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APPENDICES 


I. Executive Summary of Findings: Implementation and 
Impact 


This report describes the evaluation of the fourth year of implementation of the five-year Chicago 
Striving Readers program.! The fourth year evaluation involved 29 treatment schools (16 from Cohort 1 
and 13 from Cohort 2) and 30 control schools (15 per cohort).? 


Variations by Component and by School in Fidelity of Implementation: 


Whole-school, Targeted and Intensive Interventions, Years | through 4 


For first and second year implementation results, it was in many cases difficult to assign evidence about 
implementation to specific intervention models because the phrasing of the data collection instruments 
often was not explicit about the context. Thus, most of the evidence reflects fidelity of implementation 
of the overall program, rather than any particular intervention model. This problem was corrected for 
Year 3, however, through modifications of data collection instruments. 


Year | Implementation Study 


Overall Program Implementation 


In Year 1, instructional technology was provided through classroom listening centers and computer 
stations (media centers). Half of Striving Readers teachers surveyed reported that they used listening 
centers at least once a week; use of media centers increased during the course of the year, with 60% of 
teachers observed during the fall of 2006 using them, and 85% of those observed in spring 2007. 


Roughly half of all teachers were observed using a whole-part-whole instructional framework, 
although these teachers only spent a moderate amount of time doing so. A majority of teachers in all 
classrooms and content areas also reported that they regularly covered at least some of the 
comprehension strategies. The majority of teachers reported adjusting instructional practices based on 
diagnostic and assessment results. Insufficient data were available to determine whether the gradual 


! The first year of this study was conducted by Learning Point Associates; the balance of the study is being conducted by 
Metis Associates. 


? Cohort 1 schools began the program in school year 2006-2007; Cohort 2 began in 2007-2008. During Year 4 (school 
year 2009-20010), two of the original Cohort 2 treatment schools had closed or became turnaround schools, as did one 
control school from each Cohort. All schools, or students who were enrolled in these schools before they closed, are 
still included in impact analyses to the extent possible. 


release model was being used, although important components of the model were observed fairly 
regularly. 


The program had a slower start using Partner Reading in the Content Area, Too (PRC2); Literacy 
Intervention Teachers (LITs) were trained and felt prepared to introduce this method, but associated 
materials were not available. Text sets were not acquired until near the end of the year and were only 
used approximately one third of the time. Finally, although two fifths of teachers felt that the program 
was very effective in helping them develop the use of partner reading for vocabulary development and 
another two fifths felt that it was at least moderately effective, fewer than one third of all observed language 
arts classes included small-group activities focusing on vocabulary. 


Targeted and Intensive Interventions 


Collaboration of teachers and LITs was fostered through initial professional development activities. By 
winter 2007, all LITs reported actively collaborating with teachers. By spring, collaboration was 
supported in about two thirds of schools through school Literacy Leadership Teams and grade-level 
teams. Targeted intervention instruction of Tier 2 students by LITs took place in 46% to 59% of the 
observed literacy blocks. Intensive intervention through the AMP after-school program, including 
guided fluency practice, was provided to Tier 3 students at all schools, with attendance averaging 81%, 
and AMP classes took place in small-group settings with an average of one teacher for every 10 students. 
However, LIT noted that some topics in AMP were tedious or of limited relevance to some students. 


Year 2 Implementation Study 


Year 2 data sources that were used to provide evidence about key program features—including staff 
interviews, teacher surveys, observations and program schedules and attendance records—were used to 
create a series of rubrics to generate ratings of program fidelity within the classroom and professional 
development models. Ona 10-point scale, on average, all schools in both cohorts were implementing 
the Striving Readers program at a medium level of overall fidelity (with an average score of 6.6) during 
the 2007—2008 program year, with no overall difference between cohorts. Among the major program 
components, the highest mean ratings were obtained for the intensive intervention, for which ratings 
approached a high fidelity level. 


Implementation of Classroom Model (Whole School, Targeted and Intensive 
Interventions) 


Within the Whole School (blended) intervention component, the highest mean fidelity ratings were 
obtained for use of the whole-part-whole instructional model and the gradual release model, both of 
which received mean fidelity scores across all schools at the “high implementing” level (averaging 8.9 
and 8.5, respectively). Implementation appeared to be the most problematic for anchoring instruction in 
comprehension, for which ratings averaged 5.2. 


3 Because of the transition to a new evaluator during the second program year, these scores could not be calculated for 
Year 1. 


Within the Targeted and Intensive interventions, the most successful program component was the 
creation of small-group settings during the after-school program: all schools in both cohorts met, or at 
least came very close to, the required 15:1 student-to-teacher ratio. Nevertheless, there was still 
considerable variability by school, and several did not reach the “high implementing” level of fidelity on 
this sub-component. Providing increased instructional time for Tier 3 students remained limited by 
irregular student attendance, and by program schedules at some schools. 


Implementation variation among schools was greatest for collaboration between the LIT and 
classroom teachers and for increased instructional time for the intensive intervention, both of which 
ranged from perfect or near perfect fidelity scores to the “low implementing” level of fidelity. Cohort 2 
schools received the lowest fidelity rating in this area among all program sub-components. Substantial 
vatiations among schools were also observed for direct vocabulary instruction and whole-part-whole 
instruction. 


Implementation of Professional Development Model 


A substantial number of respondents to school leader interviews cited the importance of the professional 
development program, particularly for helping them integrate literacy into other content areas. Teacher 
sutvey respondents also expressed a desire for more training and support. However, the professional 
development component was rated at a “medium implementing” level of fidelity for most schools (5.5 
overall), implying that principals, LITs, and/or teachers had low average attendance rates at least some of 
the professional development sessions, with lower attendance rates among Cohort 1 schools. The lower 
attendance rates for Cohort 1 schools may have been due to the fact that professional development 
activities for Year 2 were not differentiated by cohort and therefore Cohort 1 staff may have felt they 
had already received sufficient training in Year 1. 


Year 3 Implementation Study 


For the Year 3 study, data sources that provided evidence of program implementation were updated to 
provide more complete and targeted data about each program component, and the implementation 
fidelity rubrics were modified accordingly. Comments on district-level program leader interviews, 
district-wide principal interviews and case study interviews, and classroom observations at case study 
schools, were used to further illuminate some of the quantitative results of the fidelity rubrics. 


Implementation of Classroom Model 


Whole School (Blended) Intervention 


All schools in both cohorts were implementing the classroom model at a medium level of overall fidelity 
during Year 3, with no notable difference between cohorts. The blended intervention model was also 
implemented at a medium level of fidelity, with higher ratings for reading comprehension strategies and 
small-group instruction. Questioning, predicting and inferring were taught most commonly, while text 


structure and metacognition were used least frequently. Small-group instruction continued to be used 
with increasing frequency throughout the program. 


Differentiated instruction varied widely among schools, although buy-in appeared to be increasing. 
Facilitating conditions included collaboration with the LIT, strong classroom management skills, 
availability of leveled materials, actively involving the students and providing supportive feedback. 
Challenges included traditional teaching habits, time constraints, large class sizes, discipline problems, 
lack of timely access to assessment results (which were more often used for “grouping” than for 
“differentiating instruction”) and limited opportunity to meet with the LIT. 


The PRC2 framework was used by the large majority of literacy teachers, but it was not used as 
regularly as expected, and was sometimes less student-focused than the model intends. Most schools 
implemented systematic instruction in academic content vocabulary using Marzano’s Vocabulary and 
Words Their Way, at a medium fidelity level, but less frequently than expected. 


Targeted Intervention 


The targeted intervention model was implemented at a high level of fidelity, on average, although more 
Cohort 1 schools had high ratings for direct instruction, while more Cohort 2 schools had higher ratings 
for teacher/LIT collaboration. Challenges to targeted intervention included different pacing needs for 
struggling readers, and time constraints aggravated by large class sizes and by LITs’ involvement in 
coaching for the whole class. 


Intensive Intervention 


AMP implementation was rated at a high fidelity level across all schools in both cohorts. Instruction in 
comprehension strategies was provided several times a week, and virtually all programs maintained a low 
student:teacher ratio. The major challenge for the intensive intervention was providing increased 
instructional time to all eligible students. Only three fifths of eligible Tier 3 students enrolled in the 
after-school class (with enrollment rates even lower at Cohort 1 schools), and attendance varied widely 
among those who did enroll. 


Purposeful Assessment and Data Driven Instruction 


The use of authentic assessments was rated at a medium fidelity level, as the use of data was beginning to 
become part of the culture in Striving Readers schools, with spelling inventories and fluency snapshots 
specifically cited as useful assessments that are easy to implement. However, Cohort 1 schools were 
somewhat more successful than Cohort 2. Assessment data were often used more to monitor individual 
progress than for broader lesson planning, and its importance for differentiated instruction was not 
always recognized. Other challenges to using assessments included the timing of the release of the data 
or the time needed to administer them. 


Materials 


Non-technology materials. Use of materials received the least favorable fidelity score of the five 
major classroom model components. Among non-technology materials, classroom libraries received 
higher ratings, but ratings for school libraries were considerably lower. Classroom libraries were used 
regularly and almost universally to support literacy instruction, and instructional staff were enthusiastic 
about their value for supporting all aspects of the program. Text sets were used to support content area 
instruction in almost all schools, and were also used in almost all literacy classrooms, although in many 
cases they were not used on a regular basis. Ratings of text sets were inconclusive, however, both 
because of the low response rates from subject-area teachers, and because of the ambiguity of whether 
sutvey respondents distinguished Striving Readers text sets from traditional collections of topical 
materials. 


Computer-assisted Instruction. LITs’ use of the Palm Pilots to support the targeted intervention 
was tated at a low level of fidelity, with one in four LITs reporting that they never used them for this 
purpose. They were often also used during AMP classes, and by literacy teachers for whole class 
instruction. Both groups used them to support vocabulary development, knowledge of key concepts and 
writing skills; LIT's also used them to teach comprehension strategies, while classroom teachers also used 
them to develop self-directed learning. Many instructors believed that students found the Palm Pilots 
engaging; however, as many still did not feel comfortable using them, and a quarter of respondents felt 
that they were not “worth the trouble.” 


Both media and listening centers were used by most literacy teachers. However, media centers were 
used even more frequently and were better integrated into the curriculum, in part because of a shortage 
of materials for the listening centers, and in part because of greater familiarity with the more traditional 
technology. 


Content Area Instruction 


Striving Readers schools seemed generally very supportive of integrating literacy into the content areas. 
There was recognition that the disciplines are mutually supportive, although this may be less true in 
content area lessons based on inquiry-driven, experiential learning. Differentiated instruction presented 
an even greater challenge in content area classes than in literacy classes. Generally, school coordinators 
felt that content teachers who are more open to less traditional techniques tended to be more successful 
at integrating literacy. 


Implementation of Professional Development Model 


Teachers and school administrators expressed appreciation for the inclusiveness and timing of the 
Striving Readers professional development program. Principals were a great help in bringing LITs on 
board with program philosophies, and as a result of LITs’ role providing feedback about teachers’ needs, 
many teachers have also found the training to be highly responsive. 


Overall fidelity scores for the professional development model were at a medium level of fidelity, but 
the fidelity of components of training that were specific to the targeted and/or intensive interventions 
was somewhat higher. Fidelity of the training was limited by two primary factors. Although most LITs 


were highly involved in weekly meetings with coordinators, attendance by teachers was problematic, and 
often not sufficient to prepare them to fully implement the model. At the same time, some staff feel that 
certain sessions have been repetitive. Leadership is still exploring ways to further strengthen its efforts 
to provide differentiated training. 


Year 4 Implementation Study and Variation in Implementation Across 
Four Years 


Implementation fidelity rubrics were again modified for the Year 4 study. Unlike the modifications that 
were implemented in Year 3, however, which were needed to account for the lack of specificity in the 
original data sources, the modifications in the fourth program year were much less substantial. In Year 
4, relatively small refinements were made to provide more precise measurements of each program 
component. As before, comments on district-level program leader interviews and district-wide principal 
interviews were used to further illuminate some of the quantitative results of the fidelity rubrics. In 
addition, case studies were conducted at six of the higher fidelity schools, where qualitative and 
quantitative data sources were analyzed to gain a comprehensive picture of each school and obtain 
insights into the factors that facilitate and hinder program implementation. 


Implementation of the Classroom Model 


All schools in both cohorts were implementing the classroom model at least at a medium level of overall 
fidelity during Year 4, including several that were implementing with high overall fidelity, showing 
progress over Years 2 and 3. Increased collaboration between project leadership and District 
Coordinators, and improved alignment between Striving Readers and other district priorities, have 
helped strengthen implementation. Nevertheless, substantial challenges remained. 


Whole School (Blended) Intervention 


Most schools implemented the blended intervention model at a medium level of fidelity. The strongest 
implementation was in teaching reading comprehension strategies, but small-group and differentiated 
instruction, and specific instructional frameworks and techniques, continued to pose a challenge. 
Nevertheless, these results also reflect progress: since the fidelity scales were first implemented in Year 2, 
the proportion of schools reporting implementation of the blended intervention at a high fidelity level 
has nearly doubled. 


Targeted Intervention 


Almost three out of four Striving Readers schools implemented the targeted intervention with high 
fidelity. Teachers and LITs formed strong collaborations, and new techniques were used project 
leadership to review and strengthen intervention strategies. However, many LITs continued to face a 
challenge to their capacity to serve the whole school. Once again, these results show that the proportion 
reporting implementation at high fidelity increased since Year 3, approaching three out of four Striving 
Readers schools during Year 4. 
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Intensive Intervention 


All schools reported implementing the intensive intervention at least at a medium fidelity level, but 
implementation was less successful than that of the targeted intervention. As with the other intervention 
models, there was at least initial improvement in implementation from Year 2 to Year 3; but in this case, 
the gains achieved during Year 3 were lost again in the fourth program year. Nevertheless, almost half 
of the schools were reported implementing the intensive intervention model with high fidelity in Year 4. 


Among the largest challenges that schools encountered was achieving high rates of enrollment and 
attendance. Schools reported being largely successful in providing instruction in comprehension, but 
were less so in fluency and vocabulary instruction. Many LITs also questioned the relevance and 
appropriateness of the AMP program for substantial numbers of students. 


Purposeful Assessment and Data Driven Instruction 


Based on teachers’, principals’ and LITs’ self-reports, some of the greatest improvements in program 
implementation have been achieved in the area of using assessment data to inform instructional 
planning: while only one in ten Striving Readers schools rated itself as using data-driven instruction with 
high fidelity during Year 2, half of all schools did so during Year 4. Teachers reported utilizing a wide 
variety of assessments beyond the state test, including informal assessments, fluency snapshots, spelling 
inventories, the Reading Benchmark Assessment and the Basic Reading Inventory. Assessment was 
conducted not merely to meet state requirements, but to actively inform the instructional program by 
monitoring students’ progress, identifying skills that need to be taught or retaught, differentiating 
instruction and creating instructional groups in class. 


Materials 


In Year 4, the use of high interest, high quality materials was still at a medium level of fidelity in most 
schools, although there was some progress since prior years in that no schools were implementing at low 
fidelity, and several had achieved high fidelity. Particular achievements were seen in the use of classroom 
libraries, which teachers were very satisfied with and reported using extensively, including to support 
grouping strategies and/or differentiation. Striving Readers text sets were also being used in social 
studies and science classrooms in most or all program schools, and content area teachers consistently 
reported finding them very useful. In addition, according to principals, technology was “thoroughly” 
integrated into the literacy curriculum at far more Striving Readers schools than control schools. Use of 
handheld computers continued to expand, but they were still used by only just over half of surveyed 
literacy teachers. They were used to support a wide variety of instructional activities in support of 
Striving Readers literacy goals, but they were generally still not being used to support differentiation 
specifically. Among the least consistently used resources were school libraries, in part because many 
schools did not have a school librarian. 


Content Area Instruction 


According to principals, literacy instruction has been integrated into content area classrooms “to a large 
extent,” especially in social studies and science classrooms, in a majority Striving Readers schools, and to 
a considerably larger degree than in control schools. Nevertheless, it remains a challenge to convince 
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some content area teachers to acknowledge the importance of incorporating literacy. At all Striving 
Readers schools, content area teachers consulted with school or external literacy experts on at least a 
monthly basis. 


Implementation of Professional Development Model 


Striving Readers continued to offer a comprehensive professional development program throughout the 
fourth program year for teachers at all grades and all content areas. Training addressed Striving Readers 

concepts, techniques and technologies, and an 18 hour course on special topics in school libraries. LIT's 

and District Coordinators continued to meet bi-weekly, and principal sessions occurred bi-monthly. 


Based on attendance rates and participants’ perceptions of the relevance of training, all schools 
implemented the professional development component with medium or high fidelity, although 
implementation was stronger among Cohort 1 schools. There has been considerable progress across 
years, possibly due to a combination of improved attendance (perhaps because the schedule has become 
increasingly more manageable and accessible in recent years), and/or increased recognition of the 
usefulness and impacts of these activities — a perception that was echoed by principals as well as teachers. 
Nevertheless, the Senior Literacy Advisor expressed concern that the training might not have brought 
staff to the level that they had hoped, in part because the district team realized in retrospect that many 
staff possessed less foundational knowledge prior to beginning Striving Readers than the model may 
have required. 


Case Study Findings 


In-depth case studies conducted at six of the higher-implementing Striving Readers schools revealed 
that, while the strategies that these schools used to address challenges to implementation were as varied 
as the circumstances that each school faced, there were also several clear themes that might provide 
insight into the fundamental characteristics necessary for success. While the specifics of implementation 
varied, these higher implementing schools shared five overarching traits: 


¢ asupportive and actively involved school administration; 
@ principals, teachers, and staff who valued and pursued professional development opportunities; 


e frequent opportunities for staff collaboration through structures such as grade-level and literacy- 
team meetings that explored instructional issues in-depth; 


e Literacy Intervention Teachers who were able to support teachers while also filling their primary 
roles as interventionists; and 


e the integration of data-driven instruction and decision making into the school’s overall literacy 
approach. 


In many respects, students at these case study schools performed better than their peers at non-case 
study program schools or at control schools. While this finding for case study schools does not come 
from a rigorous impact study and can not therefore be interpreted as evidence of program impact, it 
does point to the possibility that the Chicago Striving Readers program is more effective at certain types 
of schools under specific conditions, and it provides considerable insight into what those conditions 
might be. It is notable that virtually all of the characteristics that were common to the six case study 
schools should be achievable by other schools as well. 
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Overall Program Impacts on Students 


When considering the combined group of all students who had been offered blended intervention 
setvices during school year 2009-2010 (all tiers combined), analyses from a randomized controlled trial 
where randomization was preserved by grouping students according to an intent-to-treat (ITT) design, 
revealed no detectable overall program impacts on reading performance. Looking specifically at groups 
of sixth graders (across all tiers) who had been offered services for either one, two or three years also did 
not reveal detectable program impacts. Among all students (all tiers) who had been offered blended 
intervention services during school year 2009-2010, sixth graders responded more positively to the 
program than students in seventh grade, and Hispanic students responded more positively than students 
who were neither Black nor Hispanic. However, the program could not be considered “effective” for 
grade 6 or for Hispanic students, since impact was not significantly greater than for their control 
counterparts. 


Impacts on Students of Targeted and Intensive Interventions 


Although there were no detectable impacts for overall program effects for all students combined, 
there was a positive and statistically significant overall impact on reading performance for one of the 
groups of students whom Striving Readers is primarily designed to support: specifically, sixth-grade Tier 
2 students who had been offered one year of whole-school and targeted intervention services. However, 
no detectable impact was found for sixth-grade Tier 3 students who had been offered one year of whole- 
school, targeted, and intensive interventions, nor were there detectable impacts for Tier 2 or Tier 3 sixth 
graders who had been offered relevant services for either two years or three years. 


Among sixth-grade Tier 3 students who had been offered one year of the whole-school, targeted and 
intensive intervention services, students who were not eligible for free or reduced-price lunch responded 
more positively to the program than those who were eligible. Among sixth-grade Tier 3 students who 
had the opportunity to participate for three years, female students responded more positively to the 
program than male students. However, the program could not be considered “effective” for students in 
either of these subgroups, because impact was not significantly greater than for their respective control 
counterparts. 


Conclusions 


Given concerns expressed in previous years’ reports that program impact was likely to have been 
diminished as a result of less than optimal program implementation, the improvements that were 
observed in several aspects of program fidelity during the fourth program year offered the possibility that 
detectable impacts might emerge, assuming that the Striving Readers program (when implemented 
properly) did have a real effect. Indeed, this possibility was born out with the finding of significant 
impacts for Tier 2 students with one year of intervention. 


There are a number of reasons why program impacts might remain undetectable for other tiers and 
cohorts. First, although notable improvements in fidelity of implementation were observed in several 


areas during Year 4, there was still a considerable amount of variability in implementation. Many 
students, therefore, still did not receive the full intervention as intended. In addition, because the 
analyses are based on intent-to-treat samples, among Tier 2 students with the opportunity to receive two 
or three years of intervention, it is likely that a larger proportion (compared with the one-year sample) 
left the program and therefore did not actually receive that amount of intervention. A similar pattern 
exists for Tier 3 students. This problem is exacerbated even further among Tier 3 students, since even 
those who remained in Striving Readers schools (and therefore at least received the targeted 
intervention) often had sporadic or no attendance in the AMP intensive intervention program. In short, 
irregular implementation, irregular attendance and program attrition all conspire to dampen the impacts 
of even the most effective model. 


In addition, as discussed in the past, similarities in the initiatives (and, potentially, in the impacts) at 
control schools can result in impacts for target students that are very real and even substantial, but 
simply are not significantly greater than those for control students. There were meaningful programmatic 
differences between these groups in Year 4 that might have provided the opportunity to test the Striving 
Readers model. However, those differences were limited. Indeed, district level professional 
development efforts around literacy and differentiated instruction have been informed by the Striving 
Readers training model, resulting in greater similarities between treatment and control schools. (This 
alignment in focus and priorities has become even more notable in Year 5.) Thus, efforts to strengthen 
Striving Readers implementation—which may thereby increase its impacts—may at the same time reduce 
the ability of the current study to defect those impacts. 


Finally, and by no means least importantly, the senior literacy advisor and other testing experts have 
expressed concern that the Illinois Standards Achievement Test (ISAT) is least reliable for students with 
the lowest literacy skills—the very students for whom Striving Readers would hope to obtain the greatest 
impacts. 


The impacts for Tier 2 students were meaningful and statistically significant, albeit modest. Given all 
of the factors just discussed that make it difficult to detect impact, this finding should still be considered 
highly notable. When considered in combination with findings from the Year 3 study implying that 
impact may be greater for Tier 3 students with higher AMP attendance, and with the observed 
associations between high fidelity schools and mote positive overall ISAT results, there is clearly reason 
to train our lens specifically on the more successful schools. This is the focus of the final study year that 
is currently underway. If this study were to provide further evidence that Chicago Striving Readers has 
positive and consistent impacts when properly implemented, it would leave the district with the question, 
How can proper implementation be ensured? The Year 5 study will attempt to shed additional light on 
this question as well. 


Il. Introduction and Study Background 


Context for the Study 


In 2005, the U.S. Department of Education’s Office of Elementary and Secondary Education issued a 
request for proposals (RFP) for programs to improve adolescent literacy. The Department’s stated goal 
was to improve the performance of struggling adolescent readers and help build a strong scientific 
research base around specific strategies to help struggling readers. In response to the RFP, the Chicago 
Public Schools (CPS) proposed the Chicago Striving Readers (SR) program. This five-year program aims 
to transform teaching and learning through a seamless, aligned approach to literacy instruction for grades 
6-8 across 31 schools. 


The framework of the Chicago Striving Readers program was designed to transform the teaching 
and learning of middle-grade comprehension-focused literacy strategies and to drive long-term, systemic 
improvement in adolescent literacy in the district through a research-based model of prevention and 
intervention. The Chicago Striving Readers framework, which is based on an analysis of data and 
research reviews, encompasses optimal instructional strategies and infrastructural support services to improve 
the reading achievement of CPS middle-grade students. The framework provides a seamless and 
unifying, yet flexible, system that redesigns how reading is taught in grades 6-8. 


The Chicago Striving Readers program uses a tiered approach to working with adolescent readers. 
At the start of the school year, students are classified into three groups: those meeting or exceeding state 
performance standards for their grade (Tier 1), struggling readers who could reach grade level with 
focused support in the classroom (Tier 2); and struggling readers who require long-term intensive 
support and customized instruction (Tier 3). All three tiers receive the benefits of a whole-school 
(“blended’’’) intervention. Tiers 2 and 3 benefit from both whole-school and targeted interventions, and 
Tier 3 students benefit from the whole-school, targeted, and intensive interventions. 


The Chicago Striving Readers program has seven original key components, including three 
intervention models: 


e¢ A whole-school (blended) intervention model consisting of reading comprehension instruction 
for all students (Tiers 1, 2, and 3) in grades 6-8, including the use of literacy instruction methods 
in subject-area classrooms other than English language arts; 


4 The blended intervention model is designed to support the needs of students at all reading levels. 


eA targeted intervention model consisting of reading comprehension instruction for struggling 


readers (Tiers 2 and 3) in grades 6-8; 


e An intensive intervention model consisting of reading comprehension instruction for struggling 
readers (Tier 3) primarily in grade 6;¢ 


e Frequent, purposeful assessment and adjustment of instruction with screening, diagnostic, and 
progress-monitoring tools, including Learning First, ISAT, Basic Reading Inventory (BRI), 
fluency snapshots, spelling inventories, teacher observation and checklists, student self- 
assessment, and student interest inventories; 


e =Data-driven instruction structured through a team-based system of leadership and support; 


e =High-quality, high-interest materials—including “text set” units featuring nonfiction texts at 
different reading levels and with varied structures and organizational features—that are 
integrated with engaging technology and audio resources; and 


e Integrated, progressive, and high-quality professional development. 


As of the third program year, the project officially added an additional key component that was 
being developed on a pilot basis during Years 1 and 2: 


e Facilitation of small-group differentiated instruction and assessment through the use of 
computer-assisted reading instruction (handheld computers). 


The literacy achievement gaps of middle-grade students in the Chicago Striving Readers program are 
an outgrowth of complex socioeconomic and educational factors endemic to large urban areas like 
Chicago. Research in adolescent literacy has established that the most common problem of adolescent 
struggling readers is that they are not able to comprehend what they read (Biancarosa & Snow, 2004). In 
the 16 schools where the Striving Readers Initiative was implemented during the first year (Cohort 1), 
more than half (53%) of the students do not meet reading standards at the end of fifth grade. Although 
this deficit is reduced to 41% by the end of eighth grade, it has already taken a large toll on student 
achievement in other subject areas. The vast majority of schools in the Chicago Striving Readers 
program are located in communities that are economically disadvantaged and racially segregated. An 
average of 90% of students in these schools receive free or reduced-price lunches, and the No Child Left 
Behind (NCLB) Title I Poverty Indices range from 49% to 68%. Geographic isolation by race and 
ethnicity compounds the pedagogical challenges of educating students with learning disabilities and 
English language learners (ELLs) who come from homes where English is not always spoken. 


5 The targeted intervention model is provided to Tier 2 and 3 students in Grades 7 and 8 to the extent that each school’s 
Literacy Intervention Teacher (LIT) is able to provide instruction at these grades. In larger schools, however, the 
larger number of sixth-grade classes does not permit the LIT sufficient time to work at the upper grades. 


6 The intensive intervention model is also provided to Tier 3 students in grades 7 and 8 by classroom teachers to the 
extent possible; however, grant funds were only sufficient to directly support this intervention for 6th graders. 


Theoretical Rationale for and Description of the Intervention Models 


No single explanation accounts for why some students struggle with reading after the fifth grade while 
others do not. Although struggling adolescent readers are often characterized by terms such as a/-risk, 
unmotivated, disenchanted, or generally unsuccessful in school hteracy tasks, teseatch suggests that these descriptors 
are secondary consequences of underlying problems, not the primary causes (Peterson et al., 2000; 
Moats, 2001). Socio-cultural, motivational, and linguistic factors may be involved to varying degrees, but 
most of the research focuses on a cognitive basis or “deprivation approach” as the major underlying problem. 
This approach assumes that students must master a stable set of tasks or milestones to qualify as 
developmentally competent readers. Below-average performance on these tasks indicates that students 
have not developed the requisite skills for reading competency at a particular grade or in a particular set 
of tasks. The assumption is that below-average performance indicates deficits in varying combinations 
of word recognition and decoding skills; language processing ability at the word, sentence, or 
conversation levels; vocabulary; background knowledge; awareness of one’s own comprehension 
processes (metacognition); and comprehension and/or study strategies (Moore, Alvermann, & 
Hinchman, 2000). 


The intervention component of the Chicago Striving Readers program is based on the deprivation 
approach—CPS identifies students who are struggling with reading, tracks the nature and state of their 
deficits on an ongoing basis, and provides intensive and targeted support within school-wide language 
arts and subject-area classes and through an extended-day class. 


Administrative Structure 


The Chicago Striving Readers Program is managed by a team of leaders in the field of literacy 
instruction. The roles of the key players on this administrative team are summarized below. 


Project Director. Ms. Elizabeth Cardenas-Lopez, who had served as one of the Striving Readers 
Project Coordinators during the first program year, was appointed as project director in April 2008. The 
project director is responsible for overseeing day-to-day project operation; maintaining communications 
and meeting regularly with staff and partners at the school and district levels; ensuring fiscal integrity and 
adherence to grant requirements; monitoring and ensuring the quality of professional development; 
managing data collection and dissemination; coordinating general scheduling; planning, developing, and 
executing presentations for local events and national conferences; and interfacing with the evaluator. 

Ms. Cardenas-Lopez also has a broader role building departmental capacity and financial sustainability by 
working with the Office of Reading and Language Arts (ORLA, formerly the Office of Literacy) to 
develop middle school curriculum, select classroom materials, assess use of diagnostic instruments, and 
support district office initiatives. As of February 2011, Ms. Cardenas-Lopez was appointed Director of 
ORLA, but also continues to serve as the Project Director for Chicago Striving Readers. 


Senior Literacy Advisor. Dr. Donna Ogle, Professor of Reading and Language at National- 
Louis University (NLU), holds the position of Senior Literacy Advisor, serving on the Striving Readers 
advisory board. Dr. Ogle is a specialist and an innovator in instructional strategies that support increased 
literacy and instructional change in schools. Her responsibilities include providing both group and 
individual leadership training for Striving Readers Coordinators and for school principals and Literacy 
Intervention Teachers (LITs), including coordinating graduate-level coursework through NLU toward 


the Illinois Reading Endorsement required of each LIT who had not already received endorsement. She 
has also provided professional development for teachers and has coordinated the offering of two courses 
for librarians at Striving Readers schools. In addition, Dr. Ogle provides a critical, ongoing advisory role 
during individual meetings with the Project Director, and frequently participates in meetings with the 
Metis evaluation team. She has also taken on a degree of responsibility for supporting the LITs’ efforts 
in coordinating literacy integration at the schools. 


School Coordinators. Through spring 2010, the program was supported by four Striving 
Readers School Coordinators—each of whom had direct responsibility for seven to eight schools. 
Coordinators provide the schools with classroom instructional guidance and support and facilitate 
instructional planning. This includes conducting school-based professional development and one-on- 
one technical assistance for both teachers and LITs, and assisting LITs with observing and modeling 
lessons for teachers, providing lesson planning support. School Coordinators mentor classroom 
teachers and literacy intervention teachers in the use of comprehension strategies and techniques and 
other program components during classroom instruction, as needed or requested by teachers and/or the 
school principal. While the Coordinators work primarily with language arts classrooms and teachers, 
they also work directly with school principals, although the extent of this aspect of their role varies 
depending on the principal’s needs. Work with principals might include one-on-one coaching and 
support, planning of school-wide professional development and conferences, and discussion of specific 
classroom needs. While the majority of their time is spent on-site at the schools, the Coordinators also 
participate in meetings with other program leaders at the district office, in addition to their interactions 
with the Senior Literacy Advisor while they are receiving training. 


Technology Coordinator and Technology Consultant. The Striving Readers program 
employed two technology specialists during the 2008-2009 school year: a technology coordinator, who is 
an employee of CPS (and had served as an LIT during Year 1); and an external technology consultant, 
employed by SchoolTech Consulting. Both technology specialists were fully involved in program 
implementation, participating in district-level team meetings, supporting the project director’s planning 
efforts, and visiting the schools. Through the 2008-2009 school year, these two specialists shared 
parallel roles, with their responsibilities divided geographically. Among their primary responsibilities 
were supporting the integration of technology into literacy instruction through curriculum planning and 
strategizing with teachers. They also played a critical role in supporting each school’s technological 
capacity by installing and troubleshooting hardware and software, and providing instruction to school 
staff on the use of the program’s technological resources. During the 2009-2010 school year, in part 
because of schools’ reduced need for technology troubleshooting, these roles were filled entirely by the 
technology coordinator. 


Literacy Intervention Teachers (LITs). Grant funds are used to support one teacher in each 
participating school (for a total of 31 LITs per year through Year 3 and 29 LIT’s in Year 4) with full-time 
responsibilities in support of the Striving Readers program. LITs were selected from among existing 
CPS teachers, coaches, and literacy specialists, who were recommended, most often, by the school 
principal. All candidates were screened and interviewed by the Striving Readers Project Director and 
School Coordinators; but in all cases, the principal made the final hiring decision. These staff provide 
targeted and intensive instruction for struggling sixth- (and, wherever possible, seventh- and eighth-) 
grade students during classroom instruction and after-school programming. The LIT is responsible for 
conducting diagnostic assessments of Tier 3 students; targeting instruction to individualized student 


needs; focusing intensively on priority weaknesses during the extended-day class; planning instruction 
with collaborating teachers; analyzing and using student data to inform instruction and collaborating with 
school-based literacy teams. The LIT is also required to attend weekly training meetings and to actively 
participate in professional development sessions. 


Targeted and Intensive Interventions 


To meet the needs of struggling readers, the Chicago Striving Readers program features a two-pronged 
approach: (1) small-group, focused instruction for Tier 2 and Tier 3 students that is blended within 
language arts and subject-area classes in the school-wide model (targeted intervention); and (2) intensive, 
strategic, targeted instruction for approximately 15—20 Tier 3 students in grades 6-8 that occurs after the 
regular school day (intensive intervention). 


The targeted intervention within the school-wide model starts as soon as possible after the students 
have been tiered and continues for the duration of the school year. It involves differentiated instruction 
and scaffolding within regular classroom settings as well as increased individualized in-class support 
during the crucial sixth-grade year, when students are laying the foundation for middle-grade and future 
academic success. These activities take place during the regular English language arts (ELA) class, while 
Tier I students continue to receive instruction from the classroom teacher according to the Whole- 
School Intervention model described below. While the sixth grade takes priority, in smaller schools 
where the number of sixth-grade classrooms does not require all of the LIT’s time, the targeted 
intervention is also provided in seventh and eighth grade ELA classes. 


The intensive intervention through the extended-day approach also begins as soon as possible after 
Tier 3 students are identified (ideally by October 1), and is intended to continue through the first week in 
May (although schedules vary from school to school, and usually end in March or April). LITs conduct 
the after-school classes for sixth-grade students, and (where school finances allow) classroom teachers 
conduct the classes for students in grades 7 and 8. Intervention at this level features one hour of 
customized instruction for small homogeneous groups on four days each week. This provides: (1) an 
additional 240 minutes each week of direct and supported instruction beyond the intervention that 
occurs during the regular school day; (2) instruction in small groups of three to four students per teacher 
(within a maximum class size of 15 students to one teacher); (3) more frequent assessment and 
adjustment of instruction; and (4) highly motivating reading materials integrated with technology and 
audio. The after-school classes for the intensive intervention are built around the Achieving Maximum 
Potential (AMP) literacy program, a complete intervention system for striving middle school students who 
are reading at the third to fifth grade reading level. The AMP reading system combines research-based 
strategies with high interest, student-selected topics and incorporates an instructional model developed 
by Dr. Timothy Shanahan (Shanahan, 2006). 


The LITs are central to the targeted intervention component of the Striving Readers program. 
These teachers divide their time among sixth-grade classrooms to assist in providing differentiated 
literacy instruction. The classroom teacher is responsible for providing the overall literacy instruction in 
the classroom, assigning grades, and collaborating with the LIT to create weekly lesson and intervention 
plans. The LITs and the sixth-grade classroom teachers, who are responsible for collaborative planning, 
meet regularly to prioritize and coordinate instruction (e.g., skill review, assessment, explicit teaching, 
responsibilities, student groupings), and in small schools, LIT's also meet with seventh- and eighth-grade 


classroom teachers. The focus of the LIT on targeted, differentiated instruction to meet the needs of all 
students within the regular classroom is of particular benefit to Tier 2 and Tier 3 students who are 
struggling, including ELL students and students with learning disabilities. The identification of specific 
literacy challenges of Tier 2 and Tier 3 students enables LITs and classroom teachers to focus literacy 
instruction on the reading comprehension and vocabulary needs of these students. The use of 
differentiated materials and monitoring of the students’ progress is critical to the targeted intervention. 


Whole-School (Blended) Intervention 


The whole-school intervention occurs within both language arts and subject-area classrooms. In many 
language arts classes, direct explicit instruction in comprehension, fluency, vocabulary, and word 
identification occurs within a research-recommended 90-minute instructional model.” This model 
facilitates the gradual release of responsibility from teacher to students within a whole group/small 
group/whole group (whole-part-whole) configuration. Research by Atwell (1998) informs the 
reader/writer workshop approach for small groups. Students start with 10-20 minutes of whole-group 
instruction; move to 60 minutes of practice and application in small groups; and conclude with 10-20 
minutes of whole-group sharing.§ During the small-group period, students rotate, as assigned, among 
three 20-minute workshop activities. 


The instructional organization of subject-area classes mirrors the language arts structure. Adaptation 
of this structure takes into account the shorter time period of 45 minutes allotted for each subject as well 
as the unique purposes of each class (e.g., social studies projects, science experiments, practice of math 
computation). Current research on reading is clearly supportive of teaching adolescents to be strategic 
(and efferent) in interacting with informational text, primary sources, and relevant fiction in subject-area 
classes where comprehension strategies are taught in the service of interpreting text, not as ends in and 
of themselves (Snow, 2002). 


Based on research suggesting that new technology-based models of teaching and learning have the 
power to dramatically improve educational outcomes (Dede, 1998), the Chicago Striving Readers program is 
funding various tools to enrich student learning during small-group rotations. Each classroom has: 


e A listening center where students can listen to audiobooks and access models of fluency; 


e =A media center with three computers and access to one classroom printer to support special 
intervention software and group or individual research; and 


7 Because of scheduling difficulties, other schools have adopted a 60-minute literacy block. 


8 While the original model recommends a 90-minute structure, due to class scheduling constraints, it is most often 
implemented within a 60-minute time frame in the Chicago Striving Readers schools. 


e Ten (10) Palm Pilots (handheld computers), which are student friendly alternatives to using 
paper and pencil for activities such as note taking, brainstorming, preparing graphic organizers, 
and taking spelling tests. Palm Pilots are particularly beneficial to students who have poor 
handwriting or motor skills or have difficulty in communicating their ideas in writing and 
organizing their work.? 


During the 2004-2005 school year, CPS collaborated with university experts to develop three 
reading instructional approaches that, after proving successful, became integral parts of the Chicago 
Striving Readers school-wide component: 


Direct and explicit vocabulary instruction. This method evolved from CPS’ work with 
Robert Marzano and the Association for Supervision and Curriculum Development (ASCD). 
Vocabulary development focuses on a list of subject-specific specialized and technical words, on which 
direct instruction occurs within regular comprehension instruction and small-group activities. This 
approach borrows from Robert Marzano’s Building Academic Vocabulary (Marzano: 2004, 2005), and is 
referred to throughout this report as “Marzano’s Vocabulary”. 


Partner reading for fluency, comprehension, and vocabulary development. 
Developed by Dr. Donna Ogle, professor at National-Louis University and senior literacy advisor for the 
Chicago Striving Readers program, this model provides an opportunity for two students to read aloud to 
each other and also apply multiple comprehension strategies within a before-during-after framework. 
This exercise, which can take place during the small-group period in both language arts and subject-area 
classes, is designed to help students increase reading fluency, improve higher order thinking skills, and 
build vocabulary. 


Word Study. As of Year 2, Striving Readers began implementation of word study guided by Bear 
and Templeton in their Words Their Way work. 


Striving Readers text sets. In addition to these instructional methods, Striving Readers text set 
units related to subject-area content in social studies, science, and math are designed to increase student 
motivation and self-directed learning and stimulate the practice of comprehension strategies through 
guided reading, partner reading, self-selected individual reading, and book club discussions. These units, 
also developed in collaboration with Dr. Ogle, feature a variety of highly engaging informational 
nonfiction texts at different reading levels and with varied text structures and organizational features. 


Targeted Schools, Grades, and Students 


During its fourth year of implementation, the Chicago Striving Readers program evaluation involved 61 
target schools, including 30 treatment schools (16 from Cohort 1 and 14 from Cohort 2) and 31 control 
schools (15 from Cohort 1 and 16 from Cohort 2). Most of these schools served grades pre-K—8 or K— 


° These computers were distributed to all Striving Readers classrooms by the end of school year 2007-2008, and were in 
use as of the 2008-2009 school year. 


8, with the exception of two Striving Readers schools in the control group that were middle schools 
serving grades 4-8. Among the original 64 schools, one of the Cohort 2 schools that was originally 
assigned as a treatment school declined to participate before the program began. Additional closures and 
restructuring! occurred during the course of the program, as summarized in Table 1 below. 


Table |: Summary of School Closures and Restructuring Among Striving Readers 
Treatment and Control Schools 


Original Schools School Closures | School Restructurings 
32 16 16 | I | | 
Gonna (Closed after | (Closed after | (Consolidated with (Turnaround after 
SY 08-09) SY 08-09) feeder after SY 07- SY 07-08) 
08) 
31 16 15 0 | 0 I 
Treatment (Closed after (Turnaround after 
SY 08-09) SY 08-09) 


Among both the treatment and control schools above, restructuring occurred as a result of low 
performance, while the schools that closed did so as a result of low enrollment and low performance. 
Although survey data, school documentation and fidelity scores were not available for these schools, the 
intent-to-treat analyses discussed in Sections IV and V included all available student achievement and 
demographic data from students who enrolled in these schools while they were participating in the study. 


Within the treatment schools, all students in regular grade 6-8 classrooms received at least the 
whole-school intervention.!! Students’ eligibility for receiving the targeted interventions, intensive 
interventions, or both was based primarily on their reading proficiency in English, as determined by their 
achievement on the previous school year’s standardized reading assessment. Along with other 
characteristics, reading test results were used to assign students to tiers that determine their eligibility for 
services. Students at the highest level of proficiency (Tier 1) were not eligible for targeted or intensive 
interventions. Students at the next lower level (Tier 2) were assigned to receive the targeted intervention. 
Finally, those at the lowest level (Tier 3) were assigned to receive the targeted and the intensive 


10 Schools that become “turnaround” schools experience a complete restructuring of their staff. This is effectively 
comparable to closing the school and opening a new school in the same building. 


1! 'The exception is special education students who are not served in regular classrooms. 


interventions. Following is a summary of the procedures for assigning students to tiers that were used 
during the first four program years.!2 


In Year 1, students were assigned to tiers based on their scores on the Stanford Achievement Test 
series 10 (SAT-10), which constitutes a portion (30 items) of the Illinois Standards Achievement Test 
(ISAT) that is administered annually in Chicago.'3 For those students with missing SAT-10 test scores, 
an individually-administered informal reading inventory, the Basic Reading Inventory or BRI (Johns, 
2008) was administered and the scores from this test were used to generate preliminary tier assignments, 
which were later modified (as necessary) as soon as SAT-10 data became available. In addition, in some 
cases, students in treatment schools with SAT-10 scores that were close to the tier cutoff values were 
also given the BRI as a check on the results of the SAT-10 assessments; those students whose BRI 
scores indicated a different reading level than the SAT-10 were re-tiered. This policy was not 
implemented in all schools, however. 


The tier definitions that CPS program staff had originally established for program Year 1 were as 
follows:!4 


e students with SAT-10 reading scores at or above grade level would be placed in Tier 1; 


e students with scores below grade level but not more than one year below grade level would be 
placed in Tier 2; and 


e students who were more than one year below grade level would be placed into Tier 3. 


Because these assessments were based on the students’ prior-year test results, “grade level” for 
incoming sixth-grade students should have been based on grade equivalents for fifth-grade students. 
However, for the 2006-2007 school year, tier assignments for these students inadvertently used grade 
equivalents for students in sixth grade.!5 As a result, students scoring below the grade equivalent of 5.7 
(more than one year below grade level according to the SAT-10 grade equivalents for sixth-grade 
students) were placed into Tier 3. However, because 5.7 represents reading at grade level for fifth-grade 
students, this means that all students scoring below grade level were offered intensive intervention 
setvices, those reading up to one year above grade level were placed into Tier 2, and those reading more 
than one year above grade level were placed into Tier 1. 


CPS program staff identified additional assignment criteria that applied to special education and 
bilingual/ELL students to ensure that all students in the Striving Readers program could benefit from 


!2 Information about Years 1 and 2 assignment procedures, which took place before Metis Associates’ involvement in 
the evaluation, was obtained from reviews of syntax files from the original evaluator, Learning Point Associates (LPA); 
databases from LPA and CPS; past memoranda from LPA, CPS, and the U.S. Department of Education; and 
conversations with CPS staff. 


'5 Additional criteria, discussed below, were applied to assign ELLs and special education students. 


'4 Grade-level equivalents were determined from SAT-10 scores based on cutoffs extracted from Table C of the Spring 
Multilevel Norms Book (Harcourt Assessments, 2003). 


'5 Memorandum from LPA dated August 22, 2007. 


the services and be capable of meeting project goals. Staff had considered omitting special education 
students with any disabilities other than learning disability from the program; however, there was no 
evidence from Year 1 memoranda, nor from tier assignment syntax files,!° that any special education 
students were systematically excluded from the tier assignment process (although special education 
students other than learning disabled were removed from the original Year 1 impact analyses). For 
English language learners, most of whom did not take the standard statewide literacy exam during the 
first two program years, the original intention was to place these students in tiers based on their results 
from the Illinois Measure of Annual Growth in English (MAGE), an assessment of language 
development for English language learners. However, IMAGE data were not made available to the 
district at the time that the Year 1 tier assignments were completed. As a result, only mainstreamed ELL 
students who were tested on the Illinois Standards Achievement Test ISAT) were systematically 
assigned to tiers, based on their SAT-10 scores, using the same criteria that were used for English- 
proficient students. However, since some schools received the IMAGE data before it was available to 
the district, teachers from those schools were able to assign ELL students to tiers based on the intended 
IMAGE benchmarks. These students were assigned as follows: students who tested as meeting or 
exceeding standards on the IMAGE were placed in Tier 1, students testing as below standards were 
placed in Tier 2, and those with an IMAGE score categorized as academic warning were placed in Tier 3. 
Other non-mainstreamed ELL students were assigned to tiers once the remaining IMAGE data became 
available. The BRI was also administered to ELL students, and their tier assignments were adjusted 
based on these results. 


For Year 2 of the program, the same tier definitions were used as in Year 1, but the application of 
grade equivalents was corrected. The originally intended tier definitions were thus achieved, with 
students at or above grade level in Tier 1, those up to one year below grade level in Tier 2, and those 
more than one year below grade level in Tier 3. The same procedures from Year 1 were also used to 
assign tiers for students with missing SAT-10 data,!” special education students, and mainstreamed ELL 
students. In addition, in Year 2 IMAGE data were available to the district at the time tier assignments 
were being made, and ELL students who were tested on the IMAGE rather than the ISAT were 
assigned to tiers based on their IMAGE results. 


In the summer before Year 3 began, district program developers realized that, according to the 
developers of the AMP program, the intensive intervention was not appropriate for students who were 
reading below the third grade level. The tiering criteria were therefore revised so that only students who 
were not more than three years below grade level were assigned to Tier 3; those with lower reading 
performance were considered ineligible for Striving Readers services. At the same time, project staff felt 
that under the previous tier definitions, the intensive intervention services provided to Tier 3 students 
wete not focused on those most in need. For this reason, staff felt that the lower cutoff for Tier 2 
should be moved down one year (to two years below grade level) in order to focus on students reading at 
lower levels. However, BRI results were inconsistent with the ISAT for a number of students, for whom 


'6 Data and syntax files for Year 1 were obtained from LPA between January and March 2008. 


'7 Tn Year 2 of the program, SAT-10 scale scores were not available at the time the tier assignments were made. Instead, 
tier assignments were based on SAT-10 national percentile ranks (NPRs), which were converted into scale scores—and 
corresponding grade equivalents—using a conversion table from the SAT-10 Technical Data Report (Harcourt 
Assessments, 2004). 


BRI scores indicated that they should have been in Tier 3, while their ISAT result (under the new cutoff) 
would have placed them in Tier 2. Because the BRI, which is not administered at control schools, 
cannot be used for tier assignments, project staff agreed with Metis’ recommendation to readjust the 
boundary between Tiers 2 and 3 upward using the ISAT scores. This ensured that at least some of the 
students who otherwise would have been assigned to Tier 2 according to their ISAT scores, but should 
have been in Tier 3 according to the BRI, would be placed in Tier 3, while still using an assignment 
criterion that could be replicated at control schools. As a compromise between providing Tier 3 services 
to more students with BRI results that were lower than their ISAT results, while trying to avoid 
providing AMP services to higher-performing students who did not really need them, the cutoff between 
Tier 2 and Tier 3 was changed to 1.5 instead of two years below grade level. The criteria for Tier 1 
remained unchanged. 


Prior to Year 3, for the purpose of tiering, grade equivalence was determined by the SAT-10 portion 
of the ISAT. For Year 3 however, SAT-10 scores were not available at the time that tier assignments 
were being made. Because grade-level equivalents were not provided for the ISAT scores, they were 
inferred for these scores by determining the ISAT scores equivalent to each SAT-10 grade-equivalent 
cutoff, as predicted using a conversion from SAT-10 to ISAT derived from a bridge study aligning the 
two measures!® (MetriTech, 2006). 


Because of changes in district policies for testing English language learners, the process for assigning 
these students to tiers also changed in Year 3. As of the 2008-2009 school year, the state replaced the 
IMAGE test of English proficiency with the Assessing Comprehension and Communication in English 
State-to-State for English Language Learners (ACCESS for ELLs) exam, which became a state 
requirement for all students of limited English-speaking ability in kindergarten through grade 12. In 
addition, while the ISAT exam had previously only been administered to mainstreamed ELL students, as 
of the 2008-2009 school year the district required it for all students. For ELLs, both ISAT and 
ACCESS results were taken into consideration for tier placement during Year 3. ELL students who had 
an ACCESS score of 3.0 or higher were assigned to tiers based on their ISAT score, using the same 
criteria as for English-proficient students. Those with an ACCESS score below 3.0 were excluded from 
the Striving Readers extended-day program because it was felt that their English proficiency would be 
too low to function in and benefit from AMP.!° 


Students receiving special education services were supported by Striving Readers as suggested by 
their Individualized Education Plan (IEP). Such students would qualify for the extended- day program 
(Tier 3) as long as they performed within the range of the tiering criteria (one and one-half to three years 
below grade level) provided that the intervention treatment did not conflict with their IEPs. As in 
previous years, all students receiving special education services who were not pulled out of the regular 
classroom received the same whole-school intervention support as other students. 


'8 Predicted ISAT score = SAT-10 score * 0.72192 - 248.02832; see the bridge study (MetriTech, 2006), Table 16. 


1° Although they were not officially considered Striving Readers participants, those ELL students who were in 
mainstreamed English classes continued to receive scaffolding and differentiated instruction from the classroom 
teacher and/or LIT as needed along with the rest of the class. 


Tiering strategies and criteria remained the same in Year 4 as in Year 3. 


The specific tier assignment criteria used for students entering grade 6 are summarized in Table 2, 
while the changes in tier cutoffs from year to year are depicted in Figure 1. Assignment criteria for 
grades 7 and 8 were exactly parallel (based on grade-level equivalents) to those for sixth-grade students. 
Generally, students at all grades were assigned to tiers at the beginning of each year based on their test 
results from the prior spring. This assignment process was the same for all students regardless of 
whether or not they were new to the school or the program. Since tier assignment was always based on 
the most recent scores, a student could (in theory) receive targeted and/or intensive intervention services 
for up to three consecutive years (grades 6 through 8), if they remain eligible according to these criteria; 
and while students did not move into or out of intervention services in the middle of a school year (with 
the exception of late enrollments), they might have moved in either direction at the beginning of any 
given year. In practice, however, participation in services beyond grade 6 depended on whether the LIT 
or another teacher was available to provide these services at the upper grades, which was more often the 
case in smaller schools. 


Tier | Assignment Criteria | Year 


Table 2: Tier Assignment Criteria for Entering 6th-Grade Students 
Program Years |-4 


| [a] 


Year 2 


| Year 3 


Grade equivalent At or above 6.7 At or above 5.7 At or above 5.7 At or above 5.7 
SAT-10 NPR®! range -- 50-99 -- -- 
SAT-10 scale score range} 2659 2643 -- -- 

Tier | 
ISAT scale score range | -- - 2216!) 2216! 
IMAGE range (ELLs) -- 2207 -- -- 
ACCESS range (ELLs) -- -- >3.0 >3.0 
Grade equivalent 5.7 up to 6.7 4.7 up to 5.7 4.2 up to 5.7 4.2 up to 5.7 
SAT-10 NPR"! range -- 37-49 -- -- 
SAT-10 scale score range} 642-658 629-642 -- -- 

Tier 2 
ISAT scale score range | -- - 202-215! 202-215! 
IMAGE range (ELLs) -- 173-206 -- -- 
ACCESS range (ELLs) -- -- >3.0 >3.0 
Grade equivalent Less than 5.7 Less than 4.7 2.7 up to 4.2 2.7 up to 4.2 
SAT-10 NPR®! range -- 1-36 -- -- 
SAT-10 scale score range} $641 $628 -- -- 

Tier 3 
ISAT scale score range | -- -- 179-201! 179-201! 
IMAGE range (ELLs) -- $172 -- -- 
ACCESS range (ELLs) -- -- >3.0 >3.0 


All special education 
students with SAT-10 
scores: same criteria as 
general education 
students. 


All special education 
students with SAT-10 
scores: same criteria as 
general education 
students. 


All special education 
students with ISAT 
scores: same criteria as 
general education 
students. 


All special education 
students with ISAT 
scores: same criteria as 
general education 
students. 


Students in special education 


ELL students with 
ACCESS scores greater 
than 3.0: same criteria as 
English proficient 
students. 


ELL students with 
ACCESS scores greater 
than 3.0: same criteria as 
English proficient 
students. 


Mainstreamed ELL 
students with SAT-10 
scores: same criteria as 
English proficient 
students. 


Mainstreamed ELL 
students with SAT-10 
scores: same criteria as 
English proficient 
students. 


Bilingual/ELL students 


All other ELL students 
assigned based on BRI 
results. 


ELL students with IMAGE 
scores: assigned based on 
cutoff scores specified 
above. 


Tier | Assignment Criteria 


Other tier assignment criteria 


Year |#! 


Students with missing 
SAT-10 data: preliminary 
assignment based on BRI 
results, adjusted once 
SAT-10 data available. 


Students who enrolled in 
the target schools 
between June and 
September 2006 were 
assigned after the second 
week of school, using the 
same criteria as above. 


Students with missing 
SAT-10 data: preliminary 
assignment based on BRI 
results, adjusted once 
SAT-10 data available. 


Students who enrolled in 
the target schools after 
the initial tier assignments 
were assigned once their 
test data became 
available, using the same 
criteria as above. 


Students who enrolled in 
the target schools after 
the initial tier assignments 
were assigned once their 
test data became 
available, using the same 
criteria as above. 


Students who enrolled in 
the target schools after 
the initial tier assignments 
were assigned once their 
test data became 
available, using the same 
criteria as above. 


fl The data for Year 1 represent the acfwa/ criteria and results. However, it is important to note that these data were not 


consistent with the é#/ended criteria, which would have used grade equivalents one year lower across all tiers. 
bl NPR = National Percentile Rank. 
Il Grade-level equivalent ranges for ISAT scale scores were inferred from predicted scores from the MetriTech bridge 
study (MetriTech, 2006). 


Figure |: 6th-Grade Tier Assignments by Program Year 


Grade Equivilent 


Year 1 


Year 2 


Program Year 


Year 3 


Year 4 


During the 2009-2010 school year, out of a total population of 3,163 students in grades 6-8 among 
the 29 Striving Readers schools and 3,226 in the 29 control schools, there were 2,895 and 2,931, 


respectively, in the “intent-to-treat” (ITT) populations2° Among these, complete data (baseline and 
posttest reading scores, tier assignments, and demographic variables used as covariates) were available 
for 2,693 Striving Reader students (93% of the ITT sample) and 2,692 control students (92% of the ITT 
sample). All program students, except for the small percentage who are served outside of the regular 
classroom, received the whole-school (blended) intervention. In addition, as a result of the tier 
assignment criteria described above, there were a total of 660 students assigned to Tier 2 who were to 
receive targeted intervention services (and 628 control students reading at the same grade levels), and an 
additional 602 students in Tier 3 who were assigned to receive both targeted and intensive intervention 
services (with 632 Tier 3 control students). These distributions of the final ITT analysis samples are 
illustrated in Appendix P; distributions by grade and other demographic characteristics of these students 
are described further in “Description of Fourth-Year Samples” in Section IV. 


Logic Model 


The logic model for Year 4 of the Chicago Striving Readers program is presented on the following page. 
This model includes both professional development and classroom model components for both the 
whole-school and the targeted and intensive interventions. Year 2 modifications from the Year 1 model 
are highlighted in yellow; additional Year 3 modifications are highlighted in blue, and Year 4 
modifications are highlighted in green. Tables 3a and 3b on the following pages summarize the changes 
in the Classroom and Professional Development models, respectively, in each year. 


°0 Intent-to-treat refers to those students initially eligible for program services, even if they do not ultimately receive 
services (e.g. because they change schools or do not come to class). Specific definitions of the ITT populations for the 
Chicago Striving Readers program are provided in Section IV below. 


INPUTS 
Professional Development 


Infrastructure 


School-level 
-Literacy intervention 
teacher 


-Literacy Teams (up to 10 
members) involving the 
principal, teachers, 
district coordinator, 
librarian and tech support 


District-Level 
Project Director 
Senior Literacy Advisor 
Departmental Support Team 
District Coordinators ( 


-Provide instructional and 
monitoring supports 
-Assist with data collection 
and conduct data analysis 
-Monitor implementation 
of AMP program 

-Field liaison to schools (7 
to 8 schools each) 


| Train the trainer | 
' model results in ! 
; the gradual release 
; of responsibility to ; 
| Principals, LITs, | 
; and teachers 
1 1 
I I 


-Summer Institute Y1, 5 days; 


-Follow-up institute days (quarterly) 


-BilWeelly (6hrs) literacy instruction 


and targeted intervention meetings 


Summer Institute 


Follow-up Institute (quarterly) Yrs. 1-4 
Site-based 
Librarian coursework: 15 candidates 


Principals 
Monthly Leadership Seminars avg 7 
sessions/yr 


SR District Coordinators 


-Meetings w/ NLU Litcy Consultants 


PD Providers 


7 
Materials 


LITs (Literacy Intervention 
Teachers) 

- Conduct diagnostic (BRI) 
assessments of Tier 3 students 2x a 


year. 


instruction to meet 
needs of tier 2 and tier 3 students 
that blends with school wide model 
- Work collaboratively with school- 
based literacy teams 
-Collaborate with teachers’ in 
instructional planning and progress 
monitoring 


Teachers 

Implement instruction model (whole- 
part-whole) 

- Introduce and embed 
comprehension strategies and 
techniques (Before/During/After, use 
of text sets) 

-Differentiate instruction 

- Identify books for classroom 
library based on student interest and 
need 


Principals 

- Build instructional capacity 
within building by supporting and 
monitoring whole school 
implementation of SR strategies 
-Provide opportunities for LIT 
and Teacher collaboration for 
instructional planning and 
progress monitoring. 

-Support AMP program 


peregrine megeareame ieee competing 


district initiatives and priorities, school buy-in, teachers’ 


capacity, and instructional leadership. 


Classroom/School Practices Intermediate Outcomes 


Increased teacher efficacy 


Intensive Intervention 
-Direct, explicit instruction in 
comprehension, vocabulary and 
fluency utilizing AMP (after 
school) for Tier 3 students. 


Targeted intervention 
- Whole school differentiated 
instruction with teacher and 
LIT pushing into classroom. 
Support for Tier 2 and Tier 3 
students 


-Provide on-going formative 
assessment 


Whole school (Grades 6-8) 
- Use of explicit instruction 
with in a gradual release of 
responsibility model 

- Focus on comprehension as 
the anchor of instruction in all 
reading domains 

- Core assessment process 
(screening, diagnostic, progress 
monitoring) 

- Implement and support school 
wide literacy teams 


-Develop and) Use classroom 


libraries -Use media centers 


Library 
Support 


Technology 
Integration 


show 
Increased student efficacy instructional 


Assumptions: 


-School culture will allow implementation for these practices 
-Mobility of teachers and students is stable 


Short Term Student and 
School Outcomes 


Teachers build’) @apacity and 


improvement in 
practices in 


Long Term Student and 
School Outcomes 


for next grade level; 


teaching reading ionic 
reading progress 


Teachers improve’ levels of 
implementation of SR core 
components: 


Illinois Standards Achievement Test 


Build a culture of 


Teachers use assessment data to plan, collaboration and data 
: and ee driven decision making 


demonstrate effective use among all stake holders 
of instructional strategies, demonstrate 


improvement in quality and frequency 
of interaction with students around 
texts 


Establish procedures and strategies 
that involve varying levels of 
explicit instruction 


Increased student motivation 


and engagement through 


whole-group lessons and_ small 


group practice. 
Students 


strategies; 
Students increase time spent in 
self-selected independent reading 


stressing content area themes and 
materials 


School wide (658) grades) 


implementation of a 
systematic intervention 
program 


Develop school infrastructure 
to provide support across 
grade levels; build literacy 
leadership teams to improve 
capacity in reading instruction, 
assessment and intervention 


Develop a comprehensive 
assessment system with 
screening, diagnostic, and 
progress monitoring 


OUTCOMES 


Tier 1-3 students demonstrate readiness 


will show mid-year and_ end-of-year 


Improved student achievement so that 
more students fall into the “meet” 
“exceed standards” categories on 


or 
the 


Table 3a: Changes to Components of Classroom Model from Year | to Year 4 


Year 2 


ITilksmyernaCeyiM\y Corelel| 
Whole School 


All 6-8 grade 
students 

All 6-8 grade 
teachers 
Literacy 
Interventionists 
(LITs) 
Principals 
Librarians 


Year | 

YA 0 Olomn Kon 004.01 OFM @ Komicred sexe) sp) 
Targeted all 6-8 grade students as 
recipients of improved teacher 
capacity 


Collaborated with principals in 
ensuring program implementation 


Built classroom and school 
libraries 


Built media centers 


Established in-school literacy 
teams 


CUPAU UT Comey PAULO): Gy ise seye) (:)) 
Targeted all 6-8 grade students as 
recipients of improved teacher 
capacity 


Collaborated with principals in 
ensuring program implementation 


Continue to build classroom and 
school libraries and add Cohort 2 


Built media centers (cohort 2) 


Established in-school literacy 
teams (cohort 2) 


Year 3 

oy PAUL Oi ro oy P40) 0m G0 cred Leo) Isp) 
Targeted all 6-8 grade students as 
recipients of improved teacher 
capacity across content areas. 
Collaborated with principals in 
ensuring program implementation 


Strengthened use of media centers, 
classroom and school libraries 


Continue building 6-8 grade 
classroom and school libraries in 
all participating schools 
Reinforced literacy teams as 
systems of change 


Introduced the use of handhelds to 
differentiate instruction in ELA 
classrooms and to improve 
struggling readers engagement and 
motivation 


Year 4 

LOyPAU OR Comey MAUR LOM PAL rel aX010) sp) 
Targeted all 6-8 grade students as 
recipients of improved teacher 
capacity across content areas 
Continued to collaborate with 
principals in ensuring program 
implementation 

Continued to build classroom and 
school libraries 


Provided 40 laptops to each Case 
Study school to further support 
technology instruction and 
differentiation 

Continued to support literacy teams 
as systems of change 


Differentiated PD support to LITs 
and all targeted teachers 


Improved support in technology 
implementation and leveraged 
teacher and LIT expertise in 
workshops and on-site 
demonstrations 


Intervention Model 


Year 1 
r¥PA0) 0Xomn Ko 0y A010 Nam @ Kelme) aero) sp) 


Year 2 


CLPAU UTM Comey PAULO: Gy ise seye) (3p) 


Year 3 
oy PAUL Oc Co oy A009 Cox ved Lee) Isp) 


Year 4 
LoyPA OL OR Ko oy PAO OM PAS Xel sore) isp) 


Targeted Began core assessment process — LITs administered diagnostic LITs Administered diagnostic LITs Administered diagnostic 
focused on use of diagnostic assessment 2x/ yr to Tier 3 assessment 2x/ yr to Tier 3 assessment 2x/ yr to Tier 3 
6" grade teachers assessment (BRI) students students students 
(7" & 8" grade LITs administered fluency LITs and ELA teachers LITs and ELA teachers 
teachers in small snapshots and spelling inventories | administered fluency snapshots and | administered fluency snapshots and 
schools) 2x/yr to all students in 6" grade spelling inventories 3x/yr to all 6-8 | spelling inventories 3x/yr to all 6-8 
Literacy ELA classrooms students in ELA classrooms students in ELA classrooms 
Interventionists LITs and classroom teachers (6" LITs and ELA teachers (6" grade) | LITs and classroom teachers (6" 
Tier 2 and Tier 3 grade) followed co-teaching model | followed co-teaching model in grade) planned targeted instruction | Improved LIT and teacher 
students in working collaboratively working collaboratively in to meet needs of Tier 2 and Tier 3 collaboration (6" & 7" grade 
planning targeted instruction students where possible) in planning and 
LITs provided targeted support to LITs provided targeted support to executing targeted instruction of 
Tier 2 and Tier 3 students during Tier 2 and Tier 3 students during LITs provided targeted support to Tier 2 and Tier 3 students 
classroom instruction classroom instruction Tier 2 and Tier 3 students during 
classroom instruction LITs provided targeted support to 
LITs used progress monitoring data | Tier 2 and Tier 3 students during 
to guide student support and classroom instruction 
targeted instruction LITs and teachers used progress 
monitoring data to guide student 
support and targeted instruction 
Intensive LITs and afterschool teachers LITs and afterschool teachers LITs and afterschool teachers LITs and afterschool teachers 
(based on number of Tier 3 (based on number of Tier 3 (based on number of Tier 3 (based on number of Tier 3 
Literacy students and principals’ choice) students and principals’ choice) students and principals’ choice) students and principals’ choice) 
Interventionists taught afterschool program and taught afterschool program and taught afterschool program and taught afterschool program and 
Afterschool teachers | planned instruction that aligned planned instruction that aligned planned instruction that aligned planned instruction that aligned 
Tier 3 students with whole school and targeted with whole school and targeted with whole school and targeted with whole school and targeted 
model model model model 


Table 3b: Changes to Components of Professional Development Model from Year | to Year 4 


Program 


A Gaayo) Cyonteutecta royal 


Year 1 
YAO) Olen coe) PA010 Nam @ Kolmrel aero) isp) 


Year 2 
7/2007 to 6/2008 (31 schools) 


Year 3 
oy PAC LO}: rom oy PA0L 019M CLO Mel a0X0) Ic) 


Year 4 


(oY PAU OLR rom oy P00 LOM PALE rel sexe) I) 


Participants/ All 6-8 Classroom Teachers All 6-8 Classroom Teachers All 6-8 Classroom Teachers All 6-8 Classroom Teachers 
Stakeholders Principals Principals Principals Principals 
Librarians Librarians Librarians Librarians 
Reading Specialists Reading Specialists Reading Specialists Reading Specialists 
16 Literacy Interventionists 31 Literacy Interventionists 30 Literacy interventionists 29 Literacy interventionists 
School Technology Coordinators School Technology Coordinators School Technology Coordinators 
Project Director Project Director 
District Coordinators District Coordinators 
Professional Summer Institute: 5 days, 6hrs/ day Summer Institute: Cohort 2:5 days, | Summer Institute: 3 days, 6 hrs/day | Summer Institute: 3 days, total 14 
Development 6 hrs/ day; Cohort 1: 3 days, 6 (both cohorts) hours (both cohorts) 
Sessions Follow-up Institutes: quarterly, hrs/day Follow-up Institutes - quarterly, Follow-up Institutes - quarterly, 
3hrs/day Follow-up Institutes: quarterly, 3hrs/day 3hrs/day 
Saturday Seminars: monthly 3hrs/day 


(approx. 4-5 total”’), 3hrs/day 
Principal Training: monthly, 3 
hrs/day 

LIT Training Sessions: weekly, 
ohrs/ day 

National-Louis university 
LIT/teacher coursework: 4 terms 


School based: Literacy Teams 


Saturday Seminars: monthly (5 
total”'), 3hrs/day 

Principal Training: monthly, 3 
hrs/day 

LIT Training Sessions: weekly, 6hrs/ 
day 

National-Louis university 
LIT/teacher coursework: 4 terms 


School based: Literacy Teams & 
Grade Level Teams 


Principal Training: bi-monthly, 3 
hrs/day?!” 

LIT Training Sessions: weekly, 6 
hrs/ day 

National-Louis university LIT/ 
teacher coursework 4 terms 
NLU-Librarian workshop: monthly 
Project Director Training: as needed 
Coordinator Training: bi-monthly™* 


School based: Literacy Teams; 
Grade Level Teams; Individual & 
Group Coaching; Study Groups; 
Cross-site Visitation 


Principal Training: bi-monthly, 3 
hrs/day 

LIT Training Sessions: bi weekly, 6 
hrs/ day 


NLU-Librarian workshop: monthly 
Coordinator Training bi-monthly 
Project Director Training: as needed 


School based: Literacy Teams; 
Grade Level Teams; Individual & 
Group Coaching; Study Groups; 
Cross-site Visitation 


21 These sessions met less frequently than indicated by the model due to scheduling conflicts. 


22 Includes 7 sessions; 5 1/2-day sessions plus 2 full day sessions. The latter included attendance at a 12 hour Principal Academy that encompassed the 2009 International Reading 


Association conference. 


23 The model also called for monthly coordinator training beginning in Year 2, as indicated in the logic model; however, this activity did not begin until Year 3. 


Implementation 


Year | 


8/2006 to 6/2007 (16 schools) 


Trained all 6-8 grade teachers on 
program components with focus on 
comprehension in all reading 
domains 


Focused support on ELA teachers — 
mainly in 6"" grade 


Supported LITs with content 
knowledge, use of assessment and 
application of program components 


Enrolled qualified LITs and teachers 
in reading coursework 


Built principals’ capacity in literacy 
and program components 


LITs and afterschool teachers 
received intensive training on use 
and implementation of AMP 
program 


Year 2 

OL PAU OPM oom oy PAULO Gon Maye) Vee) fs) 
Trained all 6-8 grade teachers with 
focus on comprehension, explicit 
and differentiated instruction and 
gradual release of responsibility 
framework 


Focused support on ELA teachers — 
mainly in 6"" grade (7 & 8 grade in 
small schools) 

Supported LITs with content 
knowledge, use of assessment, 
application of program components, 
and knowledge of struggling 
adolescent learners 


Enrolled qualified LITs and teachers 
in reading coursework 


Built principals’ capacity in literacy 
and program components 


Cohort 1LITs and afterschool 
teachers received follow up training 
on use and implementation of AMP 
program 


Cohort 2 LITs and afterschool 
teachers received intensive training 
on use and implementation of AMP 
program 


Began technology training 
(including use of GoKnows) to 
LITs, 6-8 grade classroom teachers, 
and technology coordinators 


Year 3 

6/2008 to 6/2009 (30 schools) 
Trained all 6-8 grade teachers with 
focus on comprehension, explicit 
and differentiated instruction and 
gradual release of responsibility 
framework across content areas 


Focused support on ALL teachers 
across content areas 


Supported LITs with content 
knowledge, use of assessment, 
application of program components, 
and knowledge of struggling 
adolescent learners 


Enrolled qualified LITs and teachers 
in reading coursework 


Built principals; capacity in literacy 
and program components 


New LITs and after school teachers 
were trained on use and 
implementation of AMP program as 
needed 


Provided customized training to 
LITs and ELA classroom teachers in 
using handhelds for teaching and 
learning and to promote struggling 
readers engagement & motivation 
General technology training for 
teachers for all technology 
resources, such as media centers and 
listening centers 


Year 4 

oy PAU Oe Romy P40) LOM @ALEe sore) I) 
Trained all 6-8 grade teachers with 
focus on comprehension, word 
study/vocabulary, explicit and 
differentiated instruction and gradual 
release of responsibility framework 
across content areas 

Focused support for all teachers 
across content areas 


Support LITs with all components of 
the program including use of 
assessment , intervention planning 
and implementation 


Enrolled qualified LITs and teachers 
in reading coursework and supported 
their completion of the program. 
Build principals’ capacity in literacy 
and program components 


New LITs and teachers were trained 
on Striving Readers components and 
AMP training was provided as 
needed. 


Provided onsite, customized training 
to LITs and teachers in using 
handhelds and media centers 
(desktop computers) for teaching 
and learning. 
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Program 


Year 1 


Year 2 


Year 3 


Year 4 


Implementation 
Professional 
Development 
Providers 


YAO) Oo KOM 0740) OFM @ Kommsred Coxe) ) 
Dr. Donna Ogle 

National-Louis University 
Various Literacy and Leadership 
Consultants 

Pearson Consultant Group 

Tim Shanahan 

Project Director 

District Coordinators 


C/PAUUTR Comey PAULO Gy lve seye) )) 
Dr. Donna Ogle 

National-Louis University 
Various Literacy and Leadership 
Consultants 

Pearson Consultant Group 

Tim Shanahan 

Project Director 

District Coordinators 

GoKnow Technology Consultants 
Donald Bear 

Doug Fisher 


6/2008 to 6/2009 (30 schools) 
Dr. Donna Ogle 

National-Louis University 
Various Literacy and Leadership 
Consultants 


Project Director 

District Coordinators 

GoKnow Technology Consultants 
Donald Bear 

Doug Fisher 


(ePAU Oro PA00 LOM PALM es Lore) kp) 

Dr. Donna Ogle, Senior Consultant 
to Striving Readers — National-Louis 
University (NLU) 


Dr. Debbie Gurvitz and Dr. Jennifer 

Berne from NLU 

Donald Bear 

Doug Fisher 

Project Director 

District Coordinators including 
Technology Coordinator & 
Tech Consultant 

Literacy Intervention Teachers 

Classroom Teachers 
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Brief Overview of Key Evaluation Design Features 


The evaluation of Chicago Striving Readers is a five-year study designed to assess the implementation 
and impact of the initiative in 31 Title I schools in the Chicago Public School District.24 Metis 
Associates, a national research and consulting firm with offices in New York City, Atlanta, and 
Philadelphia, was selected by CPS to take over the evaluation in January 2008.?5 


The study examines the implementation and impact of both the whok-school (blended) intervention model 
and the /argeted and intensive intervention models. The research questions and key design features of the 
evaluation are summarized below.”¢ 


Research Questions 


Year 4 of the evaluation used data from a variety of sources to examine the following categories of 
research questions (the specific research questions are provided in Sections IV and V below): 


|. Overall Program Impact on Student Achievement 


e What is the overall impact of the Chicago Striving Readers program on all participating students’ 
reading scores? 


e Is there a differential overall impact of the Chicago Striving Readers program on the reading 
scores of students at different grades? 


e Is there a differential impact of the Chicago Striving Readers program on the reading scores of 
students in grades 6-8 in different NCLB subgroups, including gender, race, socioeconomic 
status, and ELL status subgroups? 


e = What is the overall impact of the Chicago Striving Readers program on the reading scores of 
students who had the opportunity to participate for two years? 


¢ What is the overall impact of the Chicago Striving Readers program on the reading scores of 
students who had the opportunity to participate for three years? 


*4’'The random assignment process for Cohort 1 schools originally placed 16 schools into the treatment group and 16 
into the control group; however, one of the treatment schools did not send a representative to program start-up 
meetings and never became part of the study. The study therefore focuses on 31 treatment schools (15 in Cohort 1 
and 16 in Cohort 2). 


°5'The transition from the original evaluator took place in January and February 2008, and the evaluation restarted in 
March of that year. 


6 These research questions have been modified slightly from Metis’ original Evaluation Design Plan of February 2008 to 
better reflect the current program status. 
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2. Program Impact on Student Achievement for Struggling Readers?’ 


What is the combined impact of the whole-school and targeted interventions of the Chicago 
Striving Readers program on Tier 2 students’ reading scores? 


What is the combined impact of the whole-school and targeted interventions of the Chicago 
Striving Readers program on the reading scores of Tier 2 students who had the opportunity to 
participate for two years? 


What is the combined impact of the whole-school and targeted interventions of the Chicago 
Striving Readers program on the reading scores of Tier 2 students who had the opportunity to 
participate for three years? 


What is the combined impact of the whole-school, targeted, and intensive interventions of the 
Chicago Striving Readers program on Tier 3 students’ reading scores? 


Is there a differential impact of the Chicago Striving Readers program on the reading scores of 
Tier 3 students in different NCLB subgroups? 


What is the combined impact of the whole-school, targeted, and intensive interventions of the 
Chicago Striving Readers program on the reading scores of Tier 3 students who had the 
Opportunity to participate for two years? 


What is the combined impact of the whole-school, targeted, and intensive interventions of the 
Chicago Striving Readers program on the reading scores of Tier 3 students who had the 
opportunity to participate for three years? 


3. Impact on Classroom Practices 


Were the Chicago Striving Readers schools different than control schools on the eight key 
program components, comprised of the classroom model (reading comprehension instruction 
for the whole-school (blended) intervention; reading comprehension instruction for the targeted 
intervention model for Tier 2 and 3 students; reading comprehension instruction for the 
intensive intervention model for Tier 3 students; purposeful assessment, data-driven instruction; 
highly motivating reading materials; and use of handheld computers) and the professional 
development model?28 


Was the Chicago Striving Readers program faithful in its implementation of the proposed 
program with regard to the eight key components? 


7 Because Tier 3 students receive both targeted and intensive interventions, and all students receive the whole-school 


intervention, it is not possible to isolate the impact of the targeted or intensive interventions, as originally proposed. 
Instead, program impact is isolated for subgroups of struggling readers. 


°8 The five original key components, which included professional development; comprehensive assessments; data-driven 


decision making; high-quality, high-interest materials; and comprehensive instruction, were redefined for the August 


2008 Implementation Executive Summary report to create these seven components. While other types of technology- 


based reading instruction were being fully implemented, implementation of the handheld computers occurred on a 
pilot basis during the 2007-2008 school year. With approval of the U.S. Department of Education, an additional key 
component for the integration of technology was officially added in Year 3, as discussed above. 
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An experimental design was established for the assessment of the research questions about program 
impact. As described further in Section IV, below, this design used random assignment at the school 
level for each of the two cohorts of schools. Cohort 1 included 32 schools that were randomly assigned 
to create equal numbers of treatment and control schools that entered the study during the 2006-2007 
school year. Cohort 2, which entered the study during school year 2007—2008, originally enrolled an 
additional 32 schools that were also randomly divided between treatment and control schools. However, 
one of the Cohort 2 schools that was designated as a treatment school declined to participate in the 
program, leaving the study with a total of 63 schools. All 63 schools include grades 6—8 among the 
grades they serve. As of the 2009-2010 school year, four of these schools (two treatment and two 
control) either closed or were restructured by the district (resulting in a complete overhaul in staff and 
students). This left the program and the study with a total of 59 schools during the fourth program year. 


The primary measures of student and teacher outcomes that were used during Year 4 of the study 
are listed below. A detailed matrix of data sources by research question is presented in Section HI, Table 


6b. 
Student Achievement: 


e ISAT -— Reading?? 


Classroom Practices and Program Fidelity: *° 
e Principal interviews 
e Expanded Literacy Improvement Survey (LIS) of teachers of English language arts3! 
e LIS surveys of LITs and non-ELA teachers? 
e AMP after-school program schedules 


e AMP attendance records 

e Principal meeting attendance records 

e Teacher professional development schedules and agendas 
e Teacher professional development attendance records 


e Program leader interviews (program director, senior literacy advisor, district coordinators and 
district technology coordinator) 


¢ Case study observations of grades 6 - 8 English language arts classes 


°° The original study design also included the Stanford Learning First ClassViews as a second measure of student 
achievement; however, these assessments were never administered. 


30 Tn addition to the listed measures, the Surveys of Enacted Curriculum (SEC) were also administered to teachers in 
treatment and control schools during Year 2. However, this extremely lengthy survey generated very low response 
rates, particularly from the control schools, and therefore could not be used in fidelity or impact analyses. Use of the 
survey was therefore eliminated from the Year 3 and 4 studies, but key topics from the SEC were incorporated into a 
revised version of the Literacy Improvement Surveys. 

3! Including self-contained teachers, lead literacy teachers, and other positions involving ELA instruction. 


32 Librarians were not surveyed during Year 4, as discussed below. 
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¢ Case study observations of grades 6 AMP classes 


¢ Case study observations of grades 6 - 8 science and social studies classes 


e Case study pre- and post-observation interviews 


e Case study principal interviews 


e Case study LIT interviews 


¢ Case study focus group interviews with classroom teachers 


Content and psychometric characteristics of the ISAT, which was used as the primary outcome 
measure for student achievement, are summarized in Table 4 SBE, 2007; DeStefano et al., 2006). 


Table 4: Characteristics of the Illinois Student Achievement Test — Reading 


Grades Metric 


Scale Scores 


Performance 
Levels are 
derived from 
the scale 
scores for 
each grade 
level. 


Reading Content 
Emphasized 


ISAT emphases are based 
on 1985 State Learning 
Goals and include: 


LC] Read with 
understanding and fluency. 


CL] Read and understand 
literature representative of 
various societies, eras, and 
ideas. 


Norming Sample and Psychometric Information 


Validity: 

For the purpose of determining Illinois students’ relative 
standing within the national population, the ISAT is equated 
to the Stanford Achievement Test — Tenth Edition (SAT — 
10). 


Correlations with the SAT — 10 exceed .94 across the six 
grades, demonstrating good convergent validity with the 
nationally accepted norm. 


Gender- and race-specific confounds were identified using 
Differential Item Functioning Analysis and systematically 
replaced until the ISAT functioned comparably for all 
individuals. 


Discriminant validity was established using Pearson 
Separation Values, which exceeded 2.5 in grades 3 through 
6 and exceeded 2.1 in grades 7 and 8. 


Reliability: 


Alpha coefficients for internal consistency, calculated using a 
parallel testing design, ranged from .86 to .91 for grades 3 
through 8. 


Inter-rater reliability scores for the open-ended response 
questions exceed 97% for all grade-level versions 
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Ill. Evaluation of the Implementation of the Whole- 


School, Targeted, and Intensive Interventions: Years | to 
4 


Study Design 


Research Questions 


The following research questions relating to assessment of the implementation of the three intervention 
models (whole-school intervention, targeted intervention, and intensive intervention) were explored 
during each of the first four years of the Chicago Striving Readers program evaluation. 


Question |: Were the Chicago Striving Readers schools different from control schools on the 
seven key components: 33 a whole-school (blended) intervention model; a targeted intervention model; an 
intensive intervention model; comprehensive assessments; data-driven decision making; high-quality, 
high-interest materials; and professional development? 


Question 2: Was the Chicago Striving Readers program faithful in its implementation of the 


proposed program with regard to the seven key components? 33 


In addition to the above, to reflect the addition of an eighth key component for computer-assisted 
reading instruction, the following research questions were added to the implementation study in Years 3 


and 4: 


Question la: Were the Chicago Striving Readers schools different from control schools on the use 
of computer-assisted reading instruction? 


Question 2a: Was the Chicago Striving Readers program faithful in its implementation of the 
proposed program with regard to the use of computer-assisted reading instruction? 


Data Sources 


Evaluation of the Chicago Striving Readers Initiative uses a mixed-method approach to obtain evidence 
of program implementation as well as program impact. This approach capitalizes on the different 


33 The original program design and research questions defined the program as having five key components: 
comprehensive assessments; data-driven decision making; high-quality, high-interest materials; comprehensive 
instruction; and professional development. During Year 2, these were redefined as the seven key components shown 
here. 
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relative advantages of qualitative and quantitative methods so that the findings from parallel measures 
can be triangulated in order to maximize confidence in the validity of the study’s conclusions. 


The data collection methods used in the first four years of the study to address research questions 
relating to assessment of program implementation were summarized in Section IT above. These data 
sources—many of which served as evidence of both fidelity of program implementation and impact on 
classroom practices—are described further below. Copies of all locally developed instruments are 


included in Appendix A. 


Surveys 


The Literacy Improvement Survey (LIS)—originally developed by Learning Point Associates (LPA)* for 
Year 1, and modified and adapted by Metis in each subsequent year to address changes in program status 
and evaluation goals—was administered each spring to all grade 6-8 English language arts teachers and 
included sets of items related to various aspects of program implementation. In Year 3, adapted versions 
of the LIS were also administered to treatment and control school librarians, as well as to the LIT's; and 
other AMP teachers, where applicable) at treatment schools. In addition, a version of the LIS survey has 
been administered to content area teachers (CATs) since Year 3, when the Striving Readers techniques 
and strategies to improve students’ literacy were expanded to social studies and science teachers. The 
LIT, librarian, and content area teacher surveys covered topics that were parallel to those addressed by 
the English Language Arts (ELA) teacher surveys, including—for LIT’ and librarians—those that were 
covered in school leader interviews in prior years. 


In Year 4, the LIS survey was administered to similar constituents as in Year 3—English language 
atts teachers, LIT’, content area teachers, and other AMP teachers, where applicable. Librarians were 
not surveyed in Year 4 because many schools did not have a librarian on staff. 


The LIS survey was modified somewhat in Year 4. Some items were removed in order to make the 
surveys less time consuming for respondents. In addition, some of the survey items were reorganized so 
that they were asked only of those staff for whom they were most pertinent. For example, the Year 4 LIS 
sutvey for ELA teachers eliminated items about the gradual release model and text sets because they are 
more applicable to the targeted and intensive interventions and content area class work, respectively, 
than to whole-school instruction in literacy classes. 


Topics addressed by the Year 4 Literacy Improvement Survey, including the survey for ELA 
teachers and its adaptations for LITs and content area teachers, included the following: 


® grouping practices; 
e = differentiation of instruction; 
e teachers’ degree of comfort in using Striving Readers techniques and materials; 


e literacy instructional purposes supported by teachers’ use of materials; 


34 LPA served as the original evaluator for the Chicago Striving Readers program, during Year 1 and the first half of 
Year 2. 
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e frequency of use, and instructional objectives supported by the use, of handheld computers; 
® organization of books in teachers’ classroom libraries; and 


e collaboration with the LIT. 


The Surveys of Enacted Curriculum (SEC)—published by LPA, the Council of Chief State School 
Officers and the Wisconsin Center for Education Research—were administered in the spring of Years 1 
and 2 to all English language arts teachers who taught students in grades 6, 7, or 8 in treatment and 
control schools. The English language arts SEC is designed to facilitate objective comparisons of what 
teachers teach with what they are expected to teach. The instrument allows classroom instruction to be 
compared to state standards, the content of the standards to be compared to the content of assessments, 
and the assessments to be compared to instruction. These comparisons are designed to measure the 
teachers’ depth of understanding of what is taught, how lessons are articulated across target grades, and 
how well instruction aligns with state content standards and state assessments.*> Because of the sheer 
length of the surveys that were administered to teachers under the original evaluation design, which may 
have adversely affected response rates, and because much of the information that was collected through 
the SEC was not directly related to research questions established for the evaluation, the use of this 
sutvey was discontinued as of Year 3. However, some topics relating to descriptions of implementation 
of program activities that had been covered by the SEC were addressed instead by the expanded Literacy 
Improvement Survey, as discussed above. 


Interviews 


In Years 1 and 2, principals, lead literacy teachers (LLTs), LITs, technology coordinators, and librarians 
in both treatment and control schools were interviewed by the independent evaluators (LPA in Year 1, 
Metis in Year 2) twice a year (fall and spring) using a structured interview protocol. In Years 3 and 4, 
principals at treatment and control schools were again interviewed by Metis staff in the spring. In Year 3, 
LITs and librarians were surveyed in lieu of interviews (as discussed above) to facilitate covering a larger 
number of issues. LIT's were surveyed in Year 4 as well. (LLTs were not surveyed or interviewed in 
Years 3 or 4, as it was found during the Year 2 interviews that many schools did not have anyone in this 
position. For the same reason, librarians were not surveyed or interviewed in Year 4.) The principal 
interviews covered topics that were parallel to those in the LIS discussed above. 


Observations 


Classroom observations conducted in Years 1 and 2 of the program used a formal observation protocol 
designed to code all observed activities in 5- to 10-minute intervals. Three waves of observations (fall, 
winter, and spring) took place during Year 1; in Year 2, as a result of the transition to a new evaluator 
midyear, only fall and spring observations were conducted. Observations were conducted in up to three 
sixth-grade English language arts classrooms in each school. The observations were conducted by 
trained observers among the evaluation staff, using the structured Adolescent Literacy Observation 


Protocol, or ALOP (see Appendix A-3). 


35 As previously explained, low response rates prevented the use of this survey in fidelity and impact analyses. 
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Case Study 


In Years 3 and 4, the original observations of classrooms across the district were replaced by a 
descriptive case study of six “high-implementation” schools, in an effort to obtain a more in-depth 
understanding of the factors that facilitate (and hinder) program implementation. The case study used 
multiple methods and respondent groups, including: observations of lessons at different grade levels and 
in different subjects; interviews with principals and LIT's; and focus groups with teachers from different 
grade levels and subject areas. All observations, interviews, and focus groups were conducted by Metis 
staff. 


The primary goals of the case study were to identify best practices and factors facilitating or 
hindering implementation both at the school level and for each main program component. The six 
schools included in the case study were selected collaboratively by Metis and district staff from among 
those considered to be stronger by program leadership, so as to provide a better opportunity to gain 
insights about best practices and conditions facilitating implementation. They were also chosen to 
represent the diversity of types of schools that are participating in the program. Program quality was 
represented by an implementation rubric, created and completed in spring 2008 by the district 
coordinators, which reflected the perceptions of the coordinators about each school’s level of 
implementation of each of several program characteristics.*° Among the higher implementing schools, 
additional selection criteria were employed to ensure that case study schools also represented both 
cohorts, reflected both large and small schools, and reflected both high and low overall performance in 
language arts (as measured by the percentage of students at or above grade level on the spring 2008 
Illinois Standards Achievement Test (ISAT) reading assessment).3” As of spring 2010, the district 
determined that program quality at three of the original six case study schools was not as strong as it had 
been. Three new high-implementing schools that had similar demographic characteristics were 
identified, and replaced these original three schools in the case study. 


In all cases, since these schools were not selected to be representative of the program as a whole, it 
was not expected that the findings would be used to identify program-wide trends. Rather, the intention 
of the case study methodology was to describe examples of implementation that, even if unique, could 
provide useful insights into best practices and potential or existing challenges encountered in the district. 


In Years 3 and 4, case study visits were conducted by Metis staff in two waves, one in the fall and 
one in the spring. Observations were conducted of ELA, science, and social studies classes. In Year 3, 
these subject areas were observed at grades 6 and 7. During Year 4, observations of eighth-grade classes 
were added to reflect the intent that the Striving Readers model would expand to progressively higher 
grades each year, and to provide the opportunity to observe classes with students who had been 


36 These included collaboration through Literacy Teams, the principal’s leadership qualities, the role of the LIT, teacher 
practices, integration of professional development, student engagement and use of Striving Readers techniques and 
strategies, and integration of technology to support implementation. 


37 Tt should be noted that, as a result of the effort to obtain variation on three variables among only six case study 
schools, the resulting group should not be assumed to include the six schools with the strongest program 
implementation, although all were considered to be implementing at a higher than average level at the time the schools 
were selected. 
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receiving SR instruction for three years. In addition, observations of the AMP after-school intensive 
intervention for sixth graders were conducted in both years. Table 5, below, shows information on the 
Year 4 case studies, including the respondent groups, data sources, and selection criteria for each school 


and wave. A description of each method used for the case study is presented after the table. 


Table 5: Year 4 Case Study Design Matrix, per Wave and School 


Grade Level | Subject Area | Data Source | Selection Criteria 
(eT xeytl >) 
6 ELA | Focus Group ee al : 
ee Variation in implementation; 
Teachers 7,8 ELA | Focus Group Availability and willingness of 
seesussecenseesausecensecssuuecsnseessueecsnsecssseecanseceanseabossvvecavvssssvvcsavvssssvvesasvecssvvesasvecssvvesasveessvvesasveessyvesssyeess4 ++ fessunessuusessunesenuseesunessuueesuuesenuesnuuessnueesnuueeenueesnuuesenuersnnne t h 
6, 7, and 8 SS and science | Focus Group pee 
6 ELA 2 Observations 
Regular classes 7 ._ 14 FLA té‘sSC~sd‘*C i Sicemade, _ High-implementing classrooms; 
(teachers and pes Presence of LIT; Availability and 
students) 8 ELA | Observation willingness of teachers 
6, 7, or 8 SS or science 2 Observations 
After-school 6 AMP iGbsenaden Availability and willingness of 
classes teachers 
LIT N/A ELA | Interview No selection; these leaders were 
interviewed in every case study 
Principal N/A N/A I Interview school 


Case study interviews and focus groups. Individual case study interviews for principals and 
LITs were conducted by Metis staff in addition to the program-wide interviews described above. The 


case study interview protocols were designed to address the same topics and themes as the program-wide 


interviews, but also to provide opportunities for respondents to elaborate on each theme in order to 


provide greater insights into factors facilitating and hindering implementation. During the fall visits, 
interview questions emphasized aspects such as planning, preparation and anticipated support, 
professional development needs, and changes with respect to the prior year. 


Focus group interviews with teachers were also designed to complement the program-wide teacher 
surveys: where the surveys focused on describing what the program looks like at each school, the 


interviews (consistent with the goals of a descriptive case study) focused more on explanations of why 


things were as they were described in the surveys. 


For both the teacher focus groups and the LIT and principal interviews, the spring interviews 


explored changes in program implementation, procedures, and policies that may have taken place at each 
school since the fall, and further explored the issues that respondents had raised in the fall to determine 
whether anticipated developments had taken place, whether identified challenges were overcome, and 
what best practices and strengths in implementation were realized. In Year 4, however, since the three 
new case study schools had not participated in the fall visits, the interviews were structured to address 
these topics more generally and to focus on changes with respect to prior years. 
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In Year 3, to interpret findings from case study interview and focus group comments, Metis used 
NVivo (version 8) software to conduct content analyses. These analyses were based on a coding 
framework that categorized responses according to the primary components of the Striving Readers 
model, valuation (successes or challenges), and respondent characteristics, and trends in the data were 
inferred from summaries of these codes. To cross-check the conceptual validity of these findings, the 
NVivo software was used to link the underlying data to summary statements about each finding, so that 
the relevance of original comments to each finding could be confirmed. A conceptually similar approach 
was used to identify themes from the Year 4 interviews, but the analyses were conducted by hand. 


Case study observations. To support the case study observations, Metis and Chicago Public 
Schools (CPS) staff collaborated to create a formal observation protocol that was designed to focus the 
collection of observation data on the techniques, strategies, frameworks, grouping structures, materials, 
and technologies that were used during the lesson. The protocol required observers to record both the 
teachet’s plans for the lesson, which were gathered through a pre-observation interview, and the actual 
observed lesson. The protocol also required observers to provide brief narrative summaries of the 
observation overall, including a description of each individual activity in the sequence in which the 
activities occurred, with concurrent small-group activities identified separately. Observers were 
instructed to address the time frame (approximate start and end times) and sequence of activities; student 
grouping structure(s); the roles of the teacher, LIT, and other adults; and the types of interactions among 
students and between students and adults. Finally, the protocol included space for the observers to 
provide a brief description of the extent to which they saw evidence that the expected characteristics of 
each instructional framework, technique, or learning strategy were present during the observed lesson. 
These expected characteristics were derived from the definitions of each technique or strategy as 
described in CPS’s Striving Readers Implementation Handbook. In Year 4, the observers also created school- 
level summaries that provided a synthesis of themes relating to each of the major program components 
that emerged across the classes observed. 


Case study analyses. For each of the six case study schools, Metis engaged in an iterative 
triangulation of qualitative findings derived from the case study group and individual interviews and the 
observational data. These school case study analyses focused on identifying the unique characteristics of 
each school that helped account for each site’s relative success in implementing the Striving Readers 
model, taking into consideration differences between respondent types, grade levels, content areas, and 
intervention models (whole school, targeted, and intensive). Particular attention was paid to findings 
relating to school context, professional development, collaboration, instructional leadership, assessment 
of progress and use of data, benefits to students, barriers to change, and outcomes of the program 
model. Data for each school were analyzed separately to create five case study reports,** while common 
themes and overarching trends across schools were analyzed to create a synthesized cross-site summary. 


Changes in the data collection process over these four years included the following: 


38 The sixth school had limited qualitative data due to its smaller size, fewer focus group participants, and logistical 
challenges that resulted in fewer observations. Because the principal was misinformed that case study findings would 
only be reported in aggregate, and these conditions made it impossible to ensure confidentiality in an individual case 
study analysis, a school-level report was not prepared for this school. However, findings from this school are included 
in the cross-site summaty. 
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e As of Year 2, documentation of program activities was maintained at the unit record level wherever 
possible, to accommodate correlational analyses of patterns of fidelity of program implementation, 
and to facilitate analyses of variations in implementation for different program components and 
under different circumstances. 

e A more targeted data collection plan was established for Year 2 (and beyond) that uses a smaller 
variety of data sources expected to have the most direct relevance to the study. The sheer volume of 
qualitative and quantitative data sources collected during Year 1 had proven onerous for some 
school personnel, and was impractical to analyze thoroughly. 

e Acase study of higher implementing schools, including classroom observations, interviews, and 
focus groups, began in Year 3, at which time the district-wide ALOP classroom observations were 
discontinued. 

e Visits to each case study school occurred in the fall and spring of Year 3. In Year 4, three of the six 
case study schools were replaced before the spring visits; case study analyses at these three schools 
are therefore based on only one wave of observations and interviews. 


Tables 6a and 6b summarize the specific data sources that were used program-wide and for the case 
study and compare those that were used in Years 1 through 4. Table 6a summarizes how the data 
sources changed between years, while Table 6b focuses on how each data source was used to address the 
research questions related to program implementation and impacts on instruction. As these tables show, 
the Chicago Striving Readers program has relied on a broad spectrum of data sources since the 
beginning of the evaluation. 
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Table 6a: Changes in Data Collection Methods from Year | to Year 4 


Data Collection Instruments and Methods 


Yr | Yr 4| Changes Over Time 

Pa ll ihe 
ALOP observation protocols were used for the district-wide 

ta | observations in Years | and 2. In Years 3 and 4, the district-wide 
Se ee observations were replaced by the case study observations and 

_- .- V V interviews. 

-~ f= Jv fv 
Protocols were modified based on observed and expected changes in 

V V V V program oversight and expansion of program leadership such as inclusion 
of technology coordinators. 
The protocols were revised to better inform the fidelity scales: most 
open-ended questions were translated into quantifiable items. Protocols 

V V V V were also expanded to include collection of data on topics such as 
integration of technology (including handheld computers), assessment 
data, and grade-level teams. 
In Year 3, LITs and Librarians were surveyed in lieu of interviews to 

V V -- -- facilitate covering more topics. Librarians were not surveyed or 
interviewed in Year 4. 
The use of this survey was discontinued as of Year 3 because of low 

V V -- -- response rates in previous years. Critical information collected through 
the SEC was incorporated into the LIS survey as of Year 3. 
The LIS was expanded in Year 3 to include topics that had been covered 

V V V V by the SEC as well as additional items about collaboration with the LIT, 
the use of materials, grouping practices, differentiation, and technology. 

d r n Years 3 and 4, LITs were surveyed in lieu of interviews to facilitate 

In Y 3 and 4, LIT din lieu of i i facili 
covering more topics. 
In Year 3, Librarians were surveyed in lieu of interviews to facilitate 

-- -- V -- covering more topics. Librarian surveys were discontinued in Year 4 
because many schools did not have a librarian. 


33 


|DY-Ye-¥m Go) | (=retu lola makina elantsvarecm-laleil i (=1uatoteny 


Yr4 


Changes Over Time 


While the district continues to maintain this documentation, it was no 


longer used for the evaluation after Year |. 


Year 2 forms were used to identify the case study schools. While the 
district still completes them each year, they were not used for the 
evaluation in Years 3 or 4. 


A documentation template was provided in Years 3 and 4 to ensure 
completeness of data (number of days attended, start and end dates, total 
enrollment, etc.). 


A documentation template was provided in Year 4 to ensure 
completeness of data. 


A documentation template was provided in Year 4 to ensure 
completeness of data. 


A documentation template was provided in Year 4 to ensure 
completeness of data. 
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Table 6b: Matrix of Research Questions and Data Collection Methods to Assess Program Implementation: 
Program Years |-4 


Data Sources (# in Target Population for Year 4 Data Collection) 


Data Collection Instruments/l NYel aoYo)!/ Dijana DYoYelllpat=\aye-tu(oy a) 


Research Questions: 
Implementation of 
Treatment and Impacts on 
Instruction 


Was the Chicago Striving Readers program faithful in its impleme 


ntation of the proposed program with regard to.../ Were the treatment schools different than control schools 


on... | 
| YI om & & & eS x . oe — s Xx | 
| ...professional Y2 ~ x Ke OR x - x ~ x | 
development? Y3 x x x 7 7 x x x ae x x oe - we 
| Y4 mM Se RM a os we we x e x x a & _ | 
| | Yl | Xx xX -- -- Xx X X X Xx -- x Xx Xx Xx Xx Xx x 
2...comprehensive Y2 0K KX = KR KR KK Ke XM &. a. de x | 
| assessments? Ge = KB xX * XX ~ = ¥ KR «= a oo & tn. Gs wd | 
oe = & « Me ee =e Sw 2 ee a. a 
| | Yl -- x | X X | X | X -- x Xx Xx Xx Xx Xx x | 
| 3...data-driven Y2 7 x x x x x oF, e x = = - fe X | 
decision making? Y3 KM SM SM 2 as OK Oe a ae a ys -- 
Y4 Xx Xx X -- -- X x -- -- -- -- -- -- -- | 
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Data Sources (# in Target Population for Year 4 Data Collection) 


Data Collection Instrumentsl@l NYel sYoXo)!/ Dijana DYoYelllpat=laye-tuloy a) 


Research Questions: 

Tan) >)(lateate-tulelame)j 
Treatment and Impacts on 
Instruction 


YD X K - = KX XK RK XK XK - XK x xX xX x xX x xX 
4....high-quality, y2 X X - =~ X X KX XK KX = = x a & a - xX 
high-interest 
materials? DS a ee ee ee, er a = = = = — 
Y4 - = KX KK KR we we KK ~ - = os 
YI X XK - = KX RK RK XR KR - XK 
5... 2 xX Me se ee RM KX MR M KH se 
comprehensive 
instruction? MS ose os 2 RR RK oe — me 
Y4 - KK KK we we KK 


ll Copies of locally developed instruments are included in Appendix A. 

bl The SEC was administered in the spring of Year 2 but the response rate was too low for the results to be useable. 

ll The LIS for teachers was administered in the spring of Years 2-4. In all three annual administrations, response rates from control schools were too low to assess 
differences in program impact on instruction between treatment and control schools, but the results were used to assess program implementation in the treatment 
schools. 
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Year | Implementation Study 


Intervention as Implemented” 


Although the program model is differentiated by design to meet the varying needs of different 
schools, classrooms, staff, and students, successful implementation of the Striving Readers model 
includes explicit expectations about the nature and frequency of implementation of certain critical 
components. The role that these key components play in moving toward programmatic goals and 
objectives was represented graphically in the logic model presented in Section I, which also provided a 
summary of the key program components. 


Tables 7 though 9 below summarize the findings, where available, regarding variations in fidelity of 
implementation of each of the first three key program components of the classroom model during Year 
1, as originally described in the Year 1 Implementation Report. It should be noted that, as was often also 
the case with evidence about implementation in Year 2 (as reported below), the nature of the data 
collection instruments made it difficult to assign evidence about implementation to specific intervention 
models, because the phrasing of these instruments often was not explicit about the context. For this 
reason, most of this evidence should be interpreted as reflecting fidelity of implementation of the overall 
Chicago Striving Readers program, rather than any particular intervention model.4° (Data that did 
provide explicit evidence relevant to particular intervention models are described in Table 8 [targeted 
intervention] and Table 9 [intensive intervention].) 


» As part of the August 2008 Executive Summary of the Year 1 Implementation Report, fidelity scales were developed 
by Metis Associates in collaboration with CPS, in order to provide a quantifiable summary of the fidelity of program 
implementation. However, because of the transition of the study to a new evaluator as of January 2008, these fidelity 
scales were not applied to data collected prior to that transition. Summaries of variations in the fidelity of program 
implementation for Year 1 that are reported here were obtained from Learning Point Associates (2007). 


As of Year 3, this problem was corrected by modifying the data collection instruments so that they address whether 
particular activities, strategies, and resources are taking place during small group instruction, during the AMP program, 


or during regular classroom activities. 
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Table 7: Variability in Implementation of Striving Readers Instructional Activities 


and Strategies: Overall 


Program Features 


PRC2; text sets and technology 
integration are used fluidly and 
alternately to support 
differentiated instruction and 
increase student motivation, 
engagement, and understanding. 


Partner Reading in the Content 
Area, Too (PRC2): a reading 
instructional framework to 
support reading comprehension 
and fluency of nonfiction text 


Text sets: high-interest books 
used to help students read 
strategically, promote 
engagement and motivation, 
and deepen their content 
knowledge 


Technology integration: 
integration of classroom 
computers, listening centers, 
and other technology materials 
designed to support small- 
group differentiated instruction 


Program Implementation 


Summary of Status of Intervention, Year | 


LITs were trained on PRC2 and reported being prepared to 
immediately introduce partner reading when possible. However, 
text materials necessary for PRC2 were unavailable in Year |. 
Therefore, content-related literacy support through this avenue had not yet 
begun. 


Text sets were widely distributed by CPS but only used about one- 
third of the time. Text sets were provided to the schools by the Striving 
Readers program, and utilized in instruction as part of the whole-school 
(blended) intervention. There was some disconnect, however, between 
availability and use of these materials: according to summaries of classroom 
observations in Year |, they were used approximately one third of the time 
(i.e., such materials were used during 33% of the “potentially relevant 
opportunities”). 


Listening centers and media centers were used fairly regularly. 
Technological resources were already implemented fairly regularly during Year 
|. Listening centers were used at least once a week by half (51%) of the 
teachers, and less than once a week but more than once a month by an 
additional 28% of teachers. Media centers were used by 60% of the teachers in 
the fall of 2006 and 85% of the teachers in the spring of 2007. 


Whole-part-whole instructional 
framework 


Roughly half of all teachers participating in the Striving Readers 
program (from 44% to 53%) were observed using a whole-part-whole 
instructional framework. On average, however, these teachers only spent 
a “moderate” amount of time doing so. Furthermore, findings were ambiguous 
as to whether entire literacy blocks were structured around this framework 
and the extent to which whole-group and small-group instructional activities 
were integrated and connected. 


Use of gradual release model to 
provide direct, explicit 
instruction and scaffold learning 
for students 


Year | data were inconclusive about use of the gradual release 
model. Insufficient data were reported from Year | to explicitly determine 
whether a comprehensive gradual release model was being used. 
Nevertheless, important components of the model were reported to have 
been observed fairly regularly. For example, coaching and scaffolding in small- 
group activities took place in more than two thirds (69%) of small-group 


activities observed in spring 2007. 
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Table 7 (continued) Variability in Implementation of Striving Readers Instructional 
Activities and Strategies: Overall Program Implementation 


Program Features 


Instruction anchor for all 
classrooms and content areas is 
focused on comprehension 


Summary of Status of Intervention, Year | 


Most teachers regularly covered at least some comprehension 
strategies. Although a majority of teachers reported that they regularly 
covered at least some of the comprehension strategies, it is unclear whether 
coverage was consistently sustained for all seven core comprehension 
strategies. What follows is the percentage of teachers reporting that they 
regularly covered each strategy: The majority of teachers reported sustained 
coverage of prediction and metacognitive strategies (74% and 63%, respectively). 
However, only about one third of respondents reported “sustained” or 
“considerable” coverage of questioning strategies and inferring (38% and 32%, 
respectively), and only 14% reported “considerable” coverage of summarization 
strategies. No information was available from the Year | reports regarding the 
degree to which teachers implemented comprehension strategies relating to 
visualization or text structure. 


Highly motivating reading 
materials integrated with 
engaging technology and audio 
resources 


Integration of text sets and technology centers was reported by 
most teachers, albeit still in the beginning stages. However, this was 
limited by delays in distribution of materials and not observed as 
often as reported. 


Although there were some delays in integrating some of the program-related 
technology, tremendous strides were made during Year |. The vast majority 
(87%) of interviewed staff reported that technology was at least somewhat 
integrated by spring 2007. However, only two fifths (40%) of those interviewed 
felt that technology was thoroughly integrated. The majority of teachers (65%) 
also reported spending at least some instructional time using computers and 
technology to learn, practice, or explore language arts, although only two fifths 
(39%) reported spending at least moderate instructional time using computers 
and technology. 


However, delays in acquisition and distribution of some materials created 
limitations. According to the Year | report, “classroom observations 
suggest... that even once materials were in place, they were not being fully 
used. Looking across three of the major types of materials targeted for use by 
the program—text sets, listening centers, and computer media centers— 
classroom observations show a use of these resources in about one third of 
the classrooms.” 
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Table 7 (cont.) Variability in Implementation of Striving Readers Instructional 
Activities and Strategies: Overall Program Implementation 


Program Features 


Frequent assessment and 
adjustment of instruction 


Summary of Status of Intervention, Year | 


The majority of teachers reported using student assessments to 
inform instruction, as well as for benchmarking and screening— 
although the assessment process was initially prolonged by delays in 
hiring LITs. The large majority (86%) of teachers in the Striving Readers 
schools reported that their instructional practices have been positively 
influenced by diagnostic and assessment results, and two thirds (66%) also 
indicated that they worked with their schools’ LLTs to “use assessment data 
for instructional planning.” A solid majority (74%) of Striving Readers teachers 
also reported that they “use assessments to directly inform and drive 
instruction,” and 76% reported that district-level tests had at least a somewhat 
positive influence on what they teach (although only 29% reported that they 
had a strong positive influence). Assessment data—including from Learning First, 
ISAT, BRI, and informal assessment—were used for a variety of instructional 
purposes, from screening and benchmarking to assessing outcomes. The 
majority (66%) of teachers, however, did not use Running Records as part of 
their instruction. Some additional setbacks to the use of assessments also 
occurred as a result of an initial delay in the hiring of the LITs from March to 
June 2006, which prevented them from receiving training (including orientation 
to the use of the BRI) prior to the start of the program and prolonged the 
initial period of assessment. 


Direct/explicit vocabulary 
instruction: 


Systematic approach to 
teaching academic content 
vocabulary in all subjects using 
Robert Marzano’s Building 
Academic Vocabulary 


Most teachers felt the program was helping them use partner 
reading for vocabulary development, but only about one quarter of 
observed classes focused on vocabulary. Approximately one fifth to one 
third of all observed English language arts classes (27%, 31%, and 19% of 
observed classes during fall, winter, and spring observations, respectively) 
included small-group activities focusing on vocabulary. Two fifths (40%) of 
surveyed teachers felt that the Striving Readers program was very effective in 
helping them develop the use of partner reading for vocabulary development, 
and almost another two fifths (38%) felt it was at least moderately effective. 
Evidence from Year | was not sufficient, however, to determine whether 
vocabulary instruction was systematic or content focused, or whether it 
specifically used Marzano’s techniques. 
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Table 8: Variability in Implementation of Striving Readers Instructional Activities 
and Strategies: Targeted Intervention 


Program Features 


Teachers and LITs collaborate 
in instructional planning and 
progress monitoring 


Summary of Status of Intervention, Year | 


LITs from all schools reported collaboration with teachers, focusing 
especially on using assessment data for grouping and to inform 
instruction. Collaborative working relationships between classroom teachers 
and LITs were fostered through initial professional development activities. 
These staff specifically collaborated on the incorporation of assessment- 
informed instructional planning; by winter 2007, all LITs reported actively 
collaborating with teachers—primarily “helping teachers use their assessment 
data to group their students and to inform and drive instruction.” Through the 
collaboration process, the role of the LITs transformed from that of advisor to 
that of a peer. For example, in the fall of 2006, the primary role of the LIT was 
to administer the assessments while in the spring of 2007, the primary role of 
the LIT was to “collaborate with teachers on curriculum.” Teachers and LITs 
also had the opportunity to collaborate through two types of instructional 
teams. Literacy Leadership Teams, which work at the school level to ensure 
the program is on track, were meeting weekly in 63% of the schools by spring 
2007, and grade-level teams were meeting weekly in 67% of the schools by this 
time. 


Increased direct and supported 
instruction: Approximately 20- 
30 minutes per day within a 60- 
to 90-minute language arts 
block 


Small group instruction by LITs occurred in about half of observed 
literacy blocks. Direct teaching of targeted youth during 20-minute periods 
of small-group instruction by LITs took place in 46% to 59% of the literacy 
blocks that were observed during Year |. However, it is likely that other 
classrooms were also using supported instruction at times other than when 
the observations took place. 


Explicit instruction in seven 
core comprehension strategies 


See Table 7, above, regarding overall focus on comprehension strategies. No 
evidence was available specific to the implementation of these strategies during 


the small-group activities. 
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Table 9: Variability in Implementation of Striving Readers Instructional Activities 
and Strategies: Intensive Intervention 


Program Features 


Increased time: 


An additional 240 minutes of 
direct and supported 
instruction beyond the 
intervention that occurs during 
the regular school day 


Summary of Status of Intervention, Year | 


AMP classes convened from October through April or May, with 
attendance averaging 81%. Additional direct and supported literacy 
instruction was provided through the AMP after-school program for Tier 3 
students. Increased time was therefore systematically provided when these 
programs were offered and when students attended. AMP classes generally 
met from October 2006 through April or early May 2007. Student attendance 
in the AMP classes throughout Year |, averaging 81%, was reasonably high but 
might not have been high enough to ensure that all Tier 3 students received 
the full weekly average of 240 minutes of additional instruction. 


Small-group setting: 


15:1 ratio of students to 
teacher 


The student:teacher ratio averaged 10:1, but some schools exceeded 
the target of 15:1. The intensive intervention took place in settings that 
exceeded the goal of an average |5:] student-teacher ratio. Across all classes, 
there was an average of one teacher for every |0 students (although some 
individual schools ran programs with more than I5 students per teacher). 


Explicit and systematic 
instruction in seven core 
comprehension strategies: 


Strategies introduced one at a 
time 


See Table 7, above, regarding overall focus on comprehension strategies. No 
evidence was available specific to the implementation of these strategies in the 
AMP classes. 


Teaching of high volume and 
depth of academic vocabulary 


See Table 7, above, regarding overall focus on vocabulary instruction. No 
evidence was available specific to the implementation of this focus in the AMP 
classes. 


Guided fluency practice 


Guided fluency practice was provided to Tier 3 students through 
vocabulary building, fluency, word identification skills, and 
background knowledge in the AMP program. However, LITs felt 
AMP was tedious or less relevant for some students. These were 
provided through interactive and diagnostic-based computer software, and the 
AMP program was implemented in every after-school program. The value of 
the AMP reading materials may have been hindered by the extent to which 
students were engaged with them: LITs noted that some of the topics were of 
limited relevance and that others became tedious for students when spread 
over several days. 


Implications for Impact Analyses 


Recognizing that systemic change is a slow process, it was notable that Striving Readers got off to a 


strong start in the first year, although some program components were not fully in place. Especially 


strong starts were seen in the AMP schedules, instruction in comprehension strategies, use of 


assessments for instructional planning, and widespread (if not fully integrated) use of technology to 


support instruction. Teachers felt ready to implement PRC2, but the necessary materials were not 


yet acquired. Among the most notable obstacles was the delay in the hiring of LITs. To the extent 


that program components were not fully integrated, it would be expected that impacts on student 


achievement would be limited. 
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Many scholars have reminded us that change is a slow process, and it is to be expected that any large, 
complex instructional program involving systemic change at the school and district level would take 
some time to get up to speed. A relevant example is found in a report by Horizon Research, Inc., which 
has been studying local systemic change initiatives in math and science for many years. Findings in that 
report addressing the likelihood of institutionalization over time indicate that it takes at least two years 
(Banilower, Boyd, Pasley, & Weiss, 2006). The report also showed the over-time trajectory of three 
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“systems” necessary for district-wide adoption: “systems for professional development,” “systems for 
aligning district policies,” and “systems for garnering and maintaining stakeholder support.” According 
to this study, these systems also do not reach equilibrium before the completion of the second year. 
Others—most notably Sarason (1971, 1996)—have found that change often takes between three and 


five years before its effects are fully felt. 


The Chicago Striving Readers Initiative is no exception. As indicated from the Year 1 findings 
summarized above, there were a number of program components that were not yet fully in place—and 
in some cases, not in place at all—during the first program year, while others had a slow start early in the 
year. This could result from delays in starting the training of LITs or in obtaining materials and 
resources, as well as from the time that teachers require to become comfortable using particular 
instructional strategies and methods. 


Nevertheless, Chicago’s Striving Readers program got off to a strong start on a number of important 
dimensions. As summarized above, the AMP schedule was fully implemented during Year 1 and, 
overall, teachers were already reporting wide use of assessments for instructional planning. Most 
teachers also reported using computers to support instruction, although in many cases technology was 
not yet thoroughly integrated. There was also widely reported use of instruction in at least some of the 
seven comprehension strategies, although this was considerably more limited for questioning and 
summarization. Although teachers reported feeling ready to implement the PRC2 instructional method, 
the necessary materials were not acquired during the first program year. Perhaps the most notable 
obstacle was the delay in the hiring of Literacy Intervention Teachers, who play such a central role in 
both the targeted and intensive instruction models, but who began their training even as they were 
beginning to fulfill their responsibilities. In addition (and possibly related to the challenge of LITs 
beginning their jobs while they were being oriented), there was limited observation of small-group 
instruction by LIT and only “moderate” time using the whole-part-whole method according to self- 
reports. Finally, although the program started on time, student attendance at AMP classes could have 
been better, and it was not clear whether enrollment was aligned with tier assignments. 


Obviously, to the extent that any program components did not get started or were not fully 
integrated from early in the year, analyses would be expected to show limited (if any) impact on student 
achievement in the first year. Similarly, for the current analyses, although the Cohort 1 students should 
have had at least as much exposure to the intervention as those in Cohort 2 during Year 2, the additional 
dosage of exposure that would normally be expected from their having started a year earlier would be 
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attenuated by these delays. As a result, it would be less likely that cohort would serve as a significant 
explanatory variable in the Year 2 analyses of overall program impact.*! 


Year 2 Implementation Study 


Intervention as Implemented 


Based on the data sources used to provide evidence about key program features, as discussed above, a 
series of rubrics were created that were used to generate scores representing level of program 
implementation.*2 In addition, interviews with district-level program leadership were used to further 
illuminate some of the quantitative results of the fidelity rubrics. Rubric scores represent the adequacy 
with which the program has been implemented for a particular school, classroom, or demographic group 
relative to the original program model. They are generated by comparing actual versus intended levels of 
implementation on factors such as the following: 


Professional development model. The proportion of targeted staff attending trainings and 


amount of different types of training received. 


Classroom model. The degree of emphasis on key instructional and assessment components, 
proportion of targeted students receiving targeted instruction, amount of time students receive intensive 
instruction (AMP after-school program attendance), availability of resources, extent of librarian support, 
and extent of integration of technology and other subject areas into literacy instruction. 


It should be noted that, depending on the source of implementation data, it was possible to calculate 
some scales at the grade, tier, or classroom level, but others could only be measured at the school level. 
However, because total scores for each major program component were derived by aggregating across 
data soutces, final scores were only calculated at the school level. All scores except the professional 
development scale were defined on a 10-point scale, where a 1 indicates that none of the key program 
characteristics are being implemented (according to observations and self-reports on surveys and 
interviews), and a 10 indicates that all key components were being implemented with the expected 
regularity. (The professional development scale starts at 0, which represents a low average rate of 
attendance at all of the key training activities.) It should be noted, however, that a 10 does not represent 
a “perfect” score. It is always theoretically possible for any school, classroom, or teacher to do more; 
however, scale values were capped at 10, which represents implementation as defined by the model. 
Further details about the fidelity scale definitions are presented in Appendix B. 


“| Because analyses of impacts specific to Tiers 2 and 3 included only sixth-grade students, this phenomenon would not 
have affected the degree to which cohort might serve as an explanatory variable in these analyses, since Year 1 
variations in dosage for grade 6 students would only affect outcomes for grade 7 students in Year 2. 


*” Because of the transition to a new evaluator during the second program year, these scores could not be calculated for 
Year 1. 


* As explained above, SEC data were not incorporated into these scales due to low response rates. 
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CPS interprets fidelity scores ranging from 8 to 10 as representing high implementation (H), scores 
above 5 but less than 8 as representing *edium implementation (M), and scores of 5 or lower as representing 
low implementation (L.). Results of the fidelity scales for Year 2 program implementation are presented in 
Tables 10a and 10b and Figure 2 below. 
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Table 10a: Results of Year 2 Implementation Fidelity Scales Mean Scores by 
Program Component 


Fidelity Component | Cohort | N | Mean Score piandard Median Eel ann Pladmaure 
Deviation Score Score Score 

Cohort | | 16 | 6.6 (M) 0.36 6.6 (M) 6.0 (M) 7.2 (M) 

Overall fidelity Cohort 2 | 15 | 6.6 (M) 0.41 6.6 (M) 5.7 (M) 7.4 (M) 
Total 31 | 6.6 (M) 0.38 6.6 (M) 5.7 (M) 7.4 (M) | 

Component |: Cohort | | 16 | 7.6 (M) 0.44 7.6 (M) 6.9 (M) 8.8 (H) 

Whole-school (blended) | Cohort2 | 15 | 7.1 (M) 0.59 7.0 (M) 5.8 (M) 8.5 (H) 

iene aen Total 31 | 7.3 (M) 0.58 7.4 (M) 5.8 (M) 8.8 (H) 
Cohort | | 16 | 9.1 (H) 0.75 9.2 (H) 7.5 (M) 10.0 (H) 

Sub-Component |: 

Whole-part-whole Cohort 2 | 15 | 8.6 (H) 1.05 8.8 (H) 6.3 (M) 10.0 (H) 
Total 31 | 8.9 (H) 0.93 9.1 (H) 6.3 (M) 10.0 (H) | 
Cohort | 16 | 8.6 (H) 0.49 8.7 (M) 7.7 (M) 9.5 (H) 

Sub-Component 2: 

eect ata aaa Cohort 2 | 15 | 8.3 (H) 0.47 8.3 (M) 7.5 (M) 9.1 (H) 
Total 31 | 8.5 (H) 0.49 8.5 (M) 7.5 (M) 9.5 (H) 
Cohort | | 16 | 5.3 (M) 0.61 5.2 (M) 4.2 (L) 6.2 (M) 

Sub-Component 3: 

Gomprdhension¥ocus Cohort 2 | 15 | 5.1 (M) 0.69 5.1 (M) 4.1 (L) 6.2 (M) 
Total 31 5.2 (M) 0.64 5.2 (M) 4.1 (L) 6.2 (M) 
Cohort | 16 | 7.3 (M) 0.85 7.4 (M) 5.9 (M) 8.6 (H) 

Sub-Component 4: PRC2 | Cohort2 | 15 | 6.8 (M) 0.91 6.4 (M) 5.6 (M) 8.2 (H) 
Total 31 7.1 (M) 0.91 7.1 (M) 5.6 (M) 8.6 (H) 
Cohort | | 16 | 7.6 (M) 1.16 7.6 (M) 4.9 (L) 9.4 (H) 

Sub-Component 5: 

Marzano’s Vocabulary Cohort 2 | 15 | 6.5 (M) 1.65 6.7 (M) 3.8 (L) 9.7 (H) 
Total Sil | || 2/0) (M4) 1.50 7.3 (M) 3.8 (L) 9.7 (H) 

Component 2: Cohort | | 16 | 5.9 (M) 1.05 5.9 (M) 3.9 (L) 7.6 (M) 

Targeted Intervention Cohort 2 | 15 | 5.9 (M) 1.02 5.9 (M) 4.2 (L) 7.8 (M) 
Total 3| 5.9 (M) 1.01 5.9 (M) 3.9 (L) 7.8 (M) 
Cohort | | 16 | 6.5 (M) 2.09 6.0 (M) 2.0 (L) 10.0 (H) 

Sub-Component 6: 

Teacher/LIT collaboration Cohort 2 | 15 | 6.7 (M) 1.79 6.0 (M) 4.0 (L) 10.0 (H) 
Total 31 | 6.6 (M) 1.92 6.0 (M) 2.0 (L) 10.0 (H) | 

Sub Conipondne7eDinect Cohort | | 16 | 5.3 (M) 0.61 5.2 (M) 4.2 (L) 6.2 (M) 

instruction in Cohort 2 | 15 | 5.1 (M) 0.69 5.1 (M) 4.1 (L) 6.2 (M) 

Reni 
Renee arte Total 31 | 5.2 (M) 0.64 5.2 (M) 4.1 (L) 6.2 (M) 
Component 3: Cohort | 16 | 8.1 (H) 0.85 8.1 (H) 6.6 (M) 9.9 (H) 
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Fidelity Component | Cohort | N | Mean Score 


Neclarerlae| Median a italiaalelan) Maximum 


Deviation Score Nelo = Nero a= 
Intensive Intervention Cohort2 | 15 | 7.5 (M) 1.30 7.5 (M) 4.9 (L) 10.0 (H) 
Total 31 | 7.8 (M) 1.13 7.9 (M) 4.9 (L) 10.0 (H) 
sie aaponene: Cohort | | 16 | 6.8 (M) 1.56 6.8 (M) 3.2 (L) 9.7 (H) 
Increased instructional Cohort 2 | 15 | 5.1 (M) 2.32 5.0 (M) 1.9 (L) 10.0 (H) 
4 
ar Total 31 | 6.0 (M) uD 6.0 (M) 1.9 (L) 10.0 (H) | 
Cohort | | 16 | 9.5 (H) 0.88 10.0 (H) 7.3 (M) 10.0 (H) 
SEP -compenene somal cane | is |e) 0.56 10.0 (H) 7.9 (M) 10.0 (H) 
group setting 
Total 31 | 9.6 (H) 0.75 10.0 (H) 7.3 (M) 10.0 (H) | 
Soninciae Cohort | | 16 | 6.9 (M) 0.78 7.0 (M) 5.7 (M) 8.6 (H) 
Purposeful Assessment & | Cohort 2 15 6.8 (M) 0.84 6.7 (M) 5.0 (L) 8.3 (H) 
Pate Gren Ince cape | Total 31 | 6.8(M) 0.80 6.8 (M) 5.0 (L) 8.6 (H) | 
Cohort | | 16 | 6.5 (M) 0.69 6.6 (M) 5.3 (M) 7.8 (M) 
eee Cohort2 | 15 | 60(M 0.62 5.9 (M 5.0 (L 72(M 
Materials a oy : sl a“ ue, 
Total 31 | 6.3 (M) 0.70 6.2 (M) 5.0 (L) 7.8 (M) 
e saivonenty Cohort | | 16 | 4.8 (L) 1.19 47 (L) 2.7 (L) 6.7 (M) 
Professional Cohort2 | 15 | 6.2 (M) 1.30 6.7 (M) 4.0 (L) 8.0 (H) 
Development Total 31 | 5.5 (M) 1.43 5.3 (M) DT) 8.0 (H) 
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Table 10b: Results of Year 2 Implementation Fidelity Scales Percent of Schools at 
Each Fidelity Level by Program Component 


C4 & C5: 


Purposeful 
Assessment 
Cl: C2: C3: nd Data C7: 
NYel sof) Blended Targeted Intensive Driven 6: Professional 
Fidelity Overall!) Intervention) Intervention Intervention Instruction Materials Development"! 


Cohort | 


Cohort 2 


High 
All Schools ; 
Combined Medium 


Low 


ll Includes professional development scores 


bl This score is based on a scale ranging from 0-10. 
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Figure 2: Results of Year 2 Implementation Fidelity Scales Mean Scores by 
Program Component by Cohort and Overall 


10.0 


Cohort 1 \ Cohort 2 BAIlI Schools 
9.0 


Medium 


Mean Fidelity Score (across schools) 
Low 


Overall [a] C1: Blended C2: Targeted C3: Intensive C4 &C5: C6: Materials C7: 
Intervention Intervention Intervention Purposeful Professional 
Assessment Development [b] 
and Data-Driven 
Instruction 


Major Program Components 


Results for the Classroom Model 


All schools in both cohorts were implementing the Striving Readers program at a medium level of 
overall fidelity during school year 2007-2008, with no overall difference between cohorts. The most 
successful components were creation of small-group settings in the intensive intervention (although 
concerns resurfaced about this program’s relevance and adaptability), whole-part-whole and the 
gradual release model. The most problematic program components included instruction in 
comprehension and increased instructional time for the AMP after-school program. 


Interpretation of variations in fidelity among different program components was hindered 
because the data sources that some scales were based on did not distinguish between whole-school, 
targeted or intensive intervention activities, and some did not fully define the program components. 
Since many ratings were based on self-report, their accuracy also depends on the respondents’ level 
of understanding of the methods that they were rating. 


Variations among schools, which were more reliable than variations among components, were 
often considerable. This was particularly true for collaboration between LITs and teachers, 
increased instructional time in AMP, direct vocabulary instruction and use of the whole-part-whole 
instructional framework. 


As the fidelity scale data show, all schools in both cohorts were implementing the Striving Readers 
program at a medium level of overall fidelity during the 2007-2008 program year, with no overall 
difference between cohorts. While variations in scores for different scales would seem to indicate that 
certain program components were implemented more successfully than others, such comparisons must 
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be interpreted with caution, since the extent to which the scales fully reflect all aspects of the program 
model varies from component to component. For example, at the component level, the highest mean 
ratings were observed for the intensive intervention (Component 3). However, the fidelity scales as 
defined only reflect the extent to which additional instructional time was provided to Tier 3 students and 
the extent to which instruction was provided in small-group settings. In addition, data sources relating 
to other aspects of program implementation were not explicit about which model they applied to. For 
example, survey questions about the use of instructional methods (such as direct vocabulary instruction) 
that are part of the intensive intervention model did not ask respondents to distinguish between their 
implementation of such activities during the regular class period or during the AMP program. Thus, the 
scores for Component 3 do not reflect either the completeness or the quality of implementation of the 
content of the intensive intervention classes. Indeed, as was also discussed in reference to the Year 1 
implementation, both LIT’s and district staff have expressed concerns about the content of the AMP 
program, which has been described as having limited relevance for some students and limited 


adaptability. 


Similar limitations restrict the extent to which Component 2 reflects the content of the targeted 
intervention. For these reasons, these scales might best be interpreted as reflecting overa// program 
fidelity.44 Fidelity of specific program components should be interpreted within the limitations of the 
focus of the data sources on which they are based. 


With these caveats about the limitations of the fidelity scales as they were defined for Year 2, the 
most successful measured program component was the creation of small-group settings during the after- 
school program—all schools in both cohorts met, or at least came very close to, the required 15:1 
student-teacher ratio. Nevertheless, there was still considerable variability by school; several schools did 
not reach the “high implementing” level of fidelity on this sub-component. 


Among the specific instructional methods that make up the Chicago Striving Readers program, the 
highest mean fidelity ratings were obtained for use of the whole-part-whole and the gradual release 
model, both of which had mean ratings across all schools at the “high implementing” level (scoring 8.9 
and 8.5, respectively, out of 10). Again, it should be noted that a major component of the original 
fidelity scale for whole-part-whole was to be based on classroom observation codes; since there was no 
clear way to interpret whether patterns of activities coded in 5-minute intervals really represented a 
whole-part-whole structure, this component was instead scored based only on LIS responses. As a 
result, both of the highest ratings were for scales for which ratings were based only on self-report. 
Although there is no reason to expect that responding teachers would consciously distort their ratings, 
their accuracy also depends on the respondents’ level of understanding of the methods that they are 


“4 The relatively limited range of fidelity scores among schools on most components also does not reflect district staffs 
impressions of the actual range of program implementation. In order to determine the extent to which such 
discrepancies may have been due in part to the design of the fidelity scales and/or the implementation rubrics used by 
program staff, fidelity scores were compared to results of the implementation rubric and to staffs impressions of 
implementation by school and by component. As a result of these comparisons, and extended conversations with the 
program director and senior literacy advisor, substantial modifications of several instruments and of the fidelity scales 
that are based on them were implemented for Year 3. These modifications are described under Data Sources above 
and in the discussion of the Year 3 Implementation Study, below. 
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rating (a concern that has been echoed by several members of the project’s district leadership team). It is 
also notable that, despite the high overall score for the whole-part-whole model, several schools were 
rated only in the “medium implementing” level of fidelity on this scale, with the lowest ratings, among 
Cohort 2 schools, at 6.9 or below (Appendix Table C-2). 


The next-highest fidelity ratings were found for implementation of the gradual release model. In this 
case again, none of the original data sources used during Year 2 provided explicit evidence for 
implementation of this model; however, the LIS did ask questions about the use of instructional 
practices that are important components of the model, including scaffolding, differentiated instruction, 
guided reading, and monitoring comprehension through questioning. Almost all schools ranked in the 
“high implementing” level of fidelity on this sub-component, while only one Cohort 1 school and three 
Cohort 2 schools fell slightly short of that mark (Appendix Table C-2). 


The program components in which implementation appeared to be the most problematic included 
anchoring instruction in comprehension (Table 10a, sub-component 3), explicit instruction in 
comprehension during small-group activities (sub-component 7), and increased instructional time for the 
after-school program (sub-component 8), with mean scores across schools of 5.2, 5.2 and 6.0, 
respectively. For both of the comprehension sub-components, five of 16 Cohort 1 schools and seven of 
15 Cohort 2 schools fell into the “low implementing” range; while for increased instructional time, two 
of 16 Cohort 1 schools and eight of 15 Cohort 2 schools fell into the “low implementing” range (see 
Appendix Tables C-2 through C-4). 


Implementation varied more dramatically from school to school on some program components than 
others. Most notable were ratings of collaboration between the LIT and classroom teachers (see 
Appendix Table C-3 for sub-component 6) and increased instructional time for the intensive 
intervention (sub-component 8), both of which ranged from perfect or near perfect scores to the low 
end of the “low implementing” level of fidelity. In addition, rather substantial variations among schools 
were observed for direct vocabulary instruction and implementation of whole-part-whole instructional 
structures (Appendix Table C-2, sub-components 5 and 1, respectively). 


District-level program staff emphasized the strengths of the program in purposeful assessment 
and data-driven instruction. While some facets of these components were not yet fully 
implemented, Striving Readers made great progress in creating a “community of learners” that use 
student assessment data to assign students to interventions, individualize instruction, conduct 
small-group activities, monitor progress, and purchase and select appropriate reading materials. 
Data-driven program management, based on LIT reflections, also increased. 


In addition to measures of implementation fidelity obtained through surveys of school-based staff, 
interviews of district-level program staff were conducted during winter 2009 in order to obtain a 
perspective from project leadership on the initiative’s implementation status during its second year. 
Among those interviewed were the program director, the senior literacy advisor, and the four district 
coordinators. Results of these interviews are discussed throughout this report. 


Additional insights into certain key program components—beyond the numerical results of the 


fidelity scales—emerged from these interviews. Although the program components of purposeful 
assessment and data-driven instruction received only medium fidelity ratings on average, a closer look 


51 


provides a clearer image of how the Striving Readers community is learning to use such data. The 
fidelity scores for this component reflect numerous facets of the efforts to use assessment data, including 
whether the use of data for instructional planning occurs through collaboration with a school literacy 
coach, during grade-level team meetings, and/or during literacy team meetings. It also reflects to what 
purposes different assessment data are used and to what extent. It is possible therefore that schools can 
be fairly strong in a number of these areas, but still receive modest fidelity scores if they are missing 
some of these facets. 


Project leadership affirm that the program was making substantial progress in major aspects of this 
objective during Year 2. They speak of creating a “community of learners” that, through the venue of 
literacy team meetings and with the critical support of the LITs, use student assessment data to assign 
students to targeted intervention models, individualize and differentiate instruction, group students for 
small-group activities (such as during whole-part-whole and PRC2 activities), determine class level 
progress, and purchase and select appropriate reading materials. This habit of mind, according to the 
coordinators, has also led to an increase in data-driven program management, using data from LIT 
reflections, which provide them “a good understanding of where more support is needed.” 


Mixed success was encountered in integrating media centers and listening centers. Many 
teachers see the benefits of this approach, but they are challenged to learn applications and adapt 
their instruction. Acquisition of audiobooks for the listening centers was delayed, but teachers also 
did not make as much use of the centers as they could have to enable students to listen to 
themselves read. 


Project leaders also spoke to the initiative’s use of technology to support instruction. Although the 
handheld computers had not yet been widely distributed until the end of Year 2 and technology use was 
not included as a formal objective until Year 3, the use of classroom media centers and listening centers 
formed a significant part of the program design from its inception. District staff explained that the 
Striving Readers schools had a mixture of successes and challenges in their use of technology during 
Year 2. Many teachers see benefits in the use of technology to support instruction, although even 
among those who do, it often poses a significant challenge in requiring them to learn the applications 
and, perhaps mote significantly, to adapt their instructional methods. 


The classroom media centers, for example, are most often used for small-group work, such as for 
conducting research on a project during group work conducted as part of whole-part-whole activities. A 
question was raised, however, as to whether these resources were being used for independent work— 
which could limit, in part, the extent to which they can be used to differentiate instruction. Use of the 
listening centers was limited somewhat during Year 2 due to delays in acquiring some of the audiobooks 
and associated software. While these delays may have temporarily limited the usefulness of the listening 
centers in helping students access models of fluency, one district coordinator pointed out that they could 
still be used to allow students to record themselves and listen to themselves read. ‘That teachers were not 
always accessing this feature of the centers was seen as an indication that teachers still needed more 
professional development in the use of these technologies. 
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Results for the Professional Development Model 


Professional development fidelity scales rated the schools on staff attendance rates in the core 
elements of training. School leaders and teachers cited professional development as a strength of 
the program. However, fidelity of professional development was rated at a ‘“‘medium 
implementing” level of fidelity, and “low implementing” for 12 schools, mostly from Cohort |, due 
to low attendance. Attendance was especially problematic at Saturday Seminars, and attendance 
problems were exacerbated by teacher turnover. An increase in the availability of on-site technical 
assistance was cited as a possible strategy that could help provide additional support for new 
teachers. 


The professional development component of the fidelity scales (Table 10a, Component 7) was also 
rated at a “medium implementing” level of fidelity for most schools; however, with a mean score of 5.5, 
it only just met this criterion, and for a number of schools—including nine of the 16 Cohort 1 schools 
and three of the 15 Cohort 2 schools—it fell into the “low implementing” range (Appendix Table C-1). 
This fidelity scale did not reflect all aspects of the initiative’s professional development program, but it 
was designed to rate the schools on their level of participation in the core elements of training. Low 
scores on this scale mean that principals, LITs, and/or teachers had low average attendance rates at at 
least some of the following training sessions: 


e LIT weekly meetings with coordinators; 

¢ = principals’ monthly professional development; 
e teachers’ summer institute; 

e teachers’ monthly Saturday seminars; and 


e teachers’ quarterly follow-up institutes. 


The project director noted that it is particularly difficult for many teachers to attend the Saturday 
seminars, and this may be a significant reason for the lower fidelity scores on this component. The 
director suggested that additional on-site assistance for classroom teachers could provide an important 
source of support to help make up for this gap. A substantial number of the respondents to the spring 
2008 school leader interviews cited the importance of the professional development program to the 
Striving Readers Initiative, particularly for helping them to integrate literacy into other content areas, and 
staff from the Striving Readers schools concurred that professional development was one of the main 
strengths of the literacy curriculum. Respondents also expressed a desire for more training and more 
support, although a number of respondents cited a lack of buy-in among teachers as a challenge to 
implementation. Attendance problems at professional development activities are also further 
exacerbated by turnover among teachers, which—although formal data could not be obtained in time for 
this report—district staff have noted is high in many Striving Readers schools. Although the 
coordinators have observed that new teachers usually are excited about the initiative, they have expressed 
concerns about the disadvantage that new teachers have due to having missed a portion of the training. 
Although the district coordinators and the LITs already strive to target their support where it is needed 
most, their efforts can be stretched thin, especially in larger schools. An increase in the availability of on- 
site technical assistance, if possible, could help provide additional support for new teachers who have not 
received the entire complement of training. A few principals commented that some of the professional 
development activities that they were expected to attend went into more detail than necessary—but on 
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the whole the comments about the Striving Readers professional development program were very 
positive, and Striving Readers schools reported more participation in training around literacy than did 
control schools. 


Because the interview data were not disaggregated by cohort, it is difficult to find an explanation in 
these data for why Cohort 1 schools had lower attendance rates. Certainly it was the intent of the 
initiative that all of these activities should continue into the second program year. It is possible that the 
lower attendance among Cohort 1 personnel reflects a feeling that they had already received sufficient 
training in some areas—or that they found the training less useful than did Cohort 2 participants. On 
the other hand, it might simply be that Cohort 1 schools are encountering greater scheduling problems. 
Given the importance of the training program to the initiative, it may be worthwhile to survey staff more 
directly about their perceptions of professional development and the factors influencing their attendance. 


Results of the Year 2 fidelity scales are presented by school in Appendix C. Additional insights into 
the fidelity and variability of program implementation were derived from interviews with principals, 
LITs, LLTs, librarians, and technology coordinators. A detailed summary of these findings, including 
school leaders’ perceptions of the professional development program, are presented in Appendix D. 


Implications for Impact Analyses 


To the extent that certain essential program components were slow in getting started—even where 
impediments occurred only at particular schools—this would again be expected to reduce the 
chances of detecting student impact. This situation was exacerbated by the lack of consistency in 
teacher training due to low attendance. 


Because several of the data sources used to measure implementation provided incomplete measures 
of fidelity, it is difficult to infer the implications of these results for program impacts. This is especially 
true regarding expected impacts of the targeted and intensive interventions, since most fidelity measures 
did not explicitly focus on program characteristics specific to these models. Nevertheless, certain 
generalizations can be made. The finding that overall program implementation, on average, was rated at 
the “medium level” of program fidelity is perhaps consistent with where one would expect a 
comprehensive five-year reform initiative to be in its second year. For this very reason, however, it is 
unlikely that the full potential of the program to affect student achievement would be observable at this 
stage. To the extent that certain essential program components, such as the focus on comprehension 
strategies, may have been slow in getting started, this would be expected to reduce the chances of 
achieving impact even further. Even where impediments to quality or dosage of implementation 
occurred only at particular schools—such as with instructional time in AMP and the extent of small- 
group instruction—it would still reduce the likelihood of being able to measure program impact, since 
these impacts are measured across all schools. 


54 


Year 3 Implementation Study 


Intervention as Implemented 


For the Year 3 study, the rubrics that were used in Year 2 to generate scores representing fidelity of 
program implementation were modified to reflect the updated data sources that were used in Year 3 (see 
Table 6b above). In addition, comments on district-level program leader interviews and district-wide 
principal interviews were used to further illuminate some of the quantitative results of the fidelity rubrics. 


Findings from the case study (staff interviews and classroom observations) were also used to obtain 
more in-depth examples of what successful implementation looked like in these six schools, as well as 
the types of challenges that they encountered and how they addressed them. As previously discussed, 
however, it is important to remember that the case study schools were not intended to be representative 
of the program as a whole. Thus, while case study findings provide important insights about Chicago 
Striving Readers, they should not be interpreted as necessarily reflecting the specific conditions 
encountered in the program as a whole. Generally, insights from the case study may be included in the 
following discussion of implementation even if they were derived from the unique experiences of a small 
number of respondents. Results are discussed in terms of multiple respondents if they were derived 
from multiple comments, or from a comment from a school leader describing a pattern believed to apply 
across teachers or classrooms within a school. The experiences of individual respondents may also be 
discussed in cases where they represent an example of, or counter example to a pattern. In a few cases, 
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discussion of findings are preceded by a qualifier (e.g., “many,” “consistent,” “clearly”). Such phrasing is 
used in reference to case study results only if the pattern applied to at least half of the individuals who 
were described or who commented about a particular topic.4> The strength of a finding is also related to 
the degree of variation among the descriptions or opinions that inform a finding: the less variation, the 
stronger the finding is. Nevertheless, even when a pattern is described as “consistent” within the case 


study, it still can not be assumed to be generalizable to all Striving Readers schools. 


The fidelity rubric scores for Year 3 were again generated by comparing actual versus intended levels 
of implementation on various factors for the professional development model and the classroom model. 
However, these scales underwent substantial modifications in Year 3 to provide a more accurate and 
complete representation of fidelity of program implementation for each main program component. 
These changes reflected modifications to LIT and teacher surveys and district-wide principal interviews 
to reflect topics that had been omitted from the original (Years 1-2) instruments, or had been previously 
addressed through the SEC, district-wide observations, or both (both were eliminated in Year 3). Some 
of the major fidelity scale revisions included the following*®: 


45 Because the open-ended nature of these interviews and observations does not explicitly prompt respondents to 
comment on particular opinions, patterns that are consistent across even a slight majority of respondents should be 
considered highly noteworthy. 


46 Plans for the Year 3 fidelity scales had also included the creation of a new component to assess fidelity of program 
implementation among content area teachers. However, the development of this component was not possible due to 
low survey response rates among this subgroup of teachers. 


55 


¢ Creating or modifying scales to reflect new or modified items that clarified the use of strategies, 
materials, resources, and assessment data specific to the blended, targeted, and intensive 
interventions. 


e Creating new sub-components within Component 5 (Materials) to separately assess fidelity in the 
use of each type of material or resource and reflect new survey items about the school library, 
teacher-librarian interactions, and handheld computers. 


e Adding items under Component 6 (Professional Development) to assess the perceptions of 
teachers, LIT's, and principals about the quality and usefulness of the Striving Readers 
professional development, and creating separate sub-components for the whole-school 
intervention vs. the targeted and intensive interventions. 


As in Year 2, fidelity scores were only calculated at the school level. All scores except the 
professional development scale were again defined on a 10-point scale, where a 1 indicates that none of 
the key program characteristics were being implemented (according to self-reports on surveys and 
interviews), and a 10 indicates that all key components were being implemented with the expected 
regularity. (The professional development scale starts at 0, which represents a low average rate of 
attendance at all of the key training activities.) As discussed for the Year 2 study, it should again be 
noted that a 10 does not represent a “perfect” score. Implementation levels for fidelity scores were 
interpreted the same way as in Year 2 (8-10 = high implementation (H), 5.1—-7.9 = medium implementation (M), 
and 0—5 = /ow implementation (L)). 


Table 11 below summarizes the major changes in the fidelity scales from Year 2 to Year 3. For a 
complete list of the Year 3 survey and interview items used in the creation of scales for each component 
and sub-component, and the formulas indicating the weights given to each item and responses to each 
item, please see Appendix E. For a detailed comparison of the changes in the scales, these Year 3 scale 
definitions can be compared to those for Year 2, which are provided in Appendix B. 
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Table ||: Changes in Fidelity Scales from Year 2 to Year 3 


Components and Sub-components Yr2 


Component |: Whole-school (blended) intervention 


Sub-component |: Whole-part-whole (Year 2)/ Small- 
group instruction (Year 3) 


Sub-Component 4: Use of PRC2 instructional 


Changes from Year 2 to Year 3 
zt 


Changed sub-component | from whole-part-whole 
to small-group instruction due to limitations in the 
ability to assess whole-part-whole. 


Refined/added/removed survey items assessing sub- 
components 2-5 (e.g., moved items about text sets 


frameworks, text sets, and technology to support V V to materials, excluded vocabulary items from 
differentiated instruction observations). 
Sub-Component 5: Marzano’s Vocabulary 
Component 2: Targeted Intervention Added survey items about the nature of 
pscoeeeennnnansseennnnnnnsieennnnnsssseennnnnsnsneennnnanusseennnnnssssseennnnnyonnsHHsseeeennnHH44eFHH0nHHH4H¢¢4H00HHHH4¢4FHH0HHHH46¢4FHH0HHH44¢0FFH00QHH46¢FFH00HHH44¢FFFH00NHH44¢84H00HHH4484F8H00NHH44¢FFH90HQHS400FFH00QHH4G005500NNHH408888HH}osssC¢¢CCCCOsssscCSCeHHsssssconennal sscssssuscsssee4ssusessseestis teachers/LIT collaboration. 
Sub-Component 6: Teacher/LIT collaboration V V 
LQ SiiiIiiItiPPIU I nionnn;;;_— Added items in the LIT survey about the use of 
Sub-Component 7: Small-group setting for Tier 2-3 practices to help students improve comprehension, 
students with direct instruction in comprehension, V V vocabulary, and fluency during the targeted 
vocabulary, and fluency Intervention. 
Component 3: Intensive Intervention V Incorporated Tier 3 sixth graders who were not 
ji cami cime amas ameanniramaiianelmesmmemacumemme smear enrolled in AMP into sub-component 8as 0% 
Sub-Component 8: Increased instructional time V attendance. 
Sub-Component 9: Small-group setting V V 
Sub-Component 10: Direct instruction in J Added items in the LIT/AMP survey to assess the 
‘ use of strategies and techniques during the intensive 
comprehension ; : 
Ei ili ere rr ee intervention. 
Sub-Component | |: Direct instruction in vocabulary V 
Pete ie eet as ac et eno 
Sub-Component |2: Direct instruction in guided r sau: ae 
2 new survey items on issues not previously 
fluency practice 
addressed. 
Component 4: Purposeful Assessment & Data- Driven a J Added sub-components about fidelity of use of 
Instruction assessments by intervention type. 
Sub-Component 13: Whole-School (Blended) F Added principal interview items about the school’s 
Intervention use of assessment data for a variety of purposes, as 
well as principal ratings about the quality of the 
Es rrr literacy and grade-level teams in using assessment 
data, under sub-component 13. 
-Component 14: Intensive Intervention V 
SHes Ses) 7 vou Added LIT/AMP sub-component |4 to reflect new 
survey items about the use of assessment data 
during the after-school program. 
Component 5: Materials I I Reorganized items to create sub-components for 


each type of material and resources used. 


Refined/added/removed survey items assessing sub- 
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Components and Sub-components Yr2 


Sub-Component | 7: Classroom libra 


Yr 3 


Changes from Year 2 to Year 3 


components |5 through 20 (e.g., removed items 
from the pre-observation checklists and the district- 
wide observations, added teacher and LIT survey 
items about the use of handheld computers and 
school libraries) 


intervention 


Sub-Component 20: Other technology resources V 
Component 6: Professional development V V Disaggregated PD component into separate sub- 
eeennnennnennnnnennnseennneennseesnnneennneennnennnneesnnnennnseennnennnseennanusseonnsennssennnssennsseonnsennssennnsenussennnssennssennnssenusseonussenusseounseenusseonuseenussenunseenusseouussenusseouuseenusseounstsnustenunetsnustenustjossseccescceessecconecennsecennecennsh s4ssssssessssesssusesss4snseste components for the whole-school intervention VS. 
Sub-Component 21: Whole-school PD \ the targeted and intensive interventions 
Sub-Component 22: PD for targeted and intensive paced teanier erase | wes aera 
V interview items about staff perceptions of the 


usefulness of PD they participated in. 


fl Because of missing data for seven of the 31 schools, this sub-component was not included in the calculations of 


Component 5 or the overall fidelity scale. 


The following discussion of implementation of the Chicago Striving Readers program during the 
2008-2009 school year is organized around the seven main components of the program model (including 
one component, integration of literacy into content area instruction, that is not included in the fidelity 
scales for reasons discussed above). Within each component, more specific findings related to the sub- 
components of the model, as well as to other topics that are important to the program but are not 


explicit model components, or that apply across more than one component, are also discussed. Results 
of the fidelity scales for Year 3 program implementation are presented in Tables 11 through 19 and 
Figures 3 through 9 (scale results by school are presented in Appendix F). These results are discussed 
below along with findings from additional survey items, interviews with program leadership, and case 
study interviews and observations that help illuminate the fidelity scale results. As previously discussed, 
case study results can not be assumed to be representative of the program as a whole, and should be 


interpreted only as illustrative examples. 


Overall Fidelity 


Results for overall fidelity scores for the classroom model are presented in Table 12 and Figure 3. 


Table 12: Results of Year 3 Classroom Model Implementation Fidelity Scales: 
Overall Fidelity 


Fidelity Component Cohort N Mean Score | Standard Median Minimum Maximum 
Deviation | Score Score Neto] a= 
Cohort | 16 7.4 (M) 5 7.3 (M) 6.8 (M) 8.5 (H) 
Overall fidelity for 
Bie easy Cohort 2 15 7.2 (M) 6 7.2 (M) 6.0 (M) 8.1 (H) 
Total Sil 7.3 (M) 5 7.2 (M) 6.0 (M) 8.5 (H) 
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Figure 3: Results of Year 3 Implementation Fidelity Scales 
Classroom Model - Overall Fidelity 
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All schools in both cohorts were implementing the classroom model at a medium level of 
overall fidelity during school year 2008-2009, with similar scores (7.4 and 7.2) for each cohort. 


As the overall fidelity scores in Table 12 above and by school in Appendix F show, all schools in 
both cohorts were implementing the classroom model of the Striving Readers program at a medium level 
of overall fidelity (with a score of 7.3 out of 10) during the 2008-2009 program year. Despite the fact 
that Cohort 2 schools had one year less experience with implementation, generally, overall fidelity scores 
(7.4 and 7.2, respectively) were similar for the two cohorts. Because the data collection instruments, and 
consequently the fidelity scales, were strengthened in Year 3 in the extent to which they reflect the 
various components of the program model, variations in scores for different scales should provide a 
more reliable representation of whether certain program components were implemented more 
successfully than others. Nevertheless, since the conversion of primarily qualitative and categorical 
information from surveys and interviews into a numeric scale is necessarily imprecise, comparisons of 
the relative success of different components are also correspondingly imprecise. Since each component 
and sub-component was measured on the same basis for all schools and classrooms, however, the results 
should provide a reliable indicator of the range of implementation among schools (or between cohorts) 
within the same component. 


Additional insights about the implementation of the Striving Readers model that were obtained from 
case study observations and interviews, district-wide principal interviews, and individual teacher and LIT 
sutvey items are discussed below in relation to each component of the model. When considering these 
summaries, it is important to bear in mind that, while case study findings are valuable in providing a 
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deeper understanding of the factors that can facilitate or hinder program implementation, they are not 


necessarily representative of the conditions encountered in the program as a whole. 


Component |: Whole-school (blended) intervention 


Fidelity scale results for the whole-school intervention are presented in Table 13 and Figure 4. 


Table 13: Results of Year 3 Classroom Model Implementation Fidelity Scales: 


Whole-school (blended) intervention 


Fidelity Component Mean Score Standard Median Minimum Maximum 
Cohort eee 
Deviation Score Score Score 
Component |: Cohort | 7.3 (M) 6.3 (M) 8.3 (H) 
Whole-school (blended) Cohort2 | 15 | 7.2 7 : 9 7.2 (M) 5.8 (M) 8.9 (H) 
intervention 
Total 31 7.2 (M) ol 7.2 (M) 5.8 (M) 8.9 (H) 
Cohort | 16 7.4 (M) 9 7.5 (M) 5.7 (M) 9.3 (H) 
ep eomponsne omalerelP Gono? 11s. 75a) ll 7.1 (M) 6.0 (M) 10.0 (H) 
instruction 
Total 31 7.5 (M) 1.0 7.5 (M) 5.7 (M) 10.0 (H) 
Cohort | 16 6.7 (M) 1.2 6.9 (M) 3.8 (L) 8.3 (H) 
Sub-Component 2: Gradual 
Cohort 2 15 6.8 (M) 1.3 6.8 (M) 4.4 (L) 8.8 (H) 
release model 
Total 31 6.8 (M) 1.2 6.8 (M) 3.8 (L) 8.8 (H) 
Cohort | 16 7.8 (M) 5 7.9 (M) 6.7 (M) 8.6 (H) 
Sub-Component 3: 
Comprehensionfocus Cohort 2 15 7.9 (M) 8 8.1 (H) 6.3 (M) 9.0 (H) 
Total 31 7.9 (M) 6 7.9 (M) 6.3 (M) 9.0 (H) 
Sub-Component 4:Use of PRC2 Cohort | 16 7.1 (M) 1.1 7.5 (M) 4.9 (L) 8.7 (H) 
nsurucuonal famewons sex Canara | is. | Ge) 13 6.0 (M) 4.6 (L) 10.0 (H) 
sets, and technology to support 
differentiated instruction Total 31 6.8 (M) |.22 7.1 (M) 4.6 (L) 10.0 (H) 
Cohort | 16 7.0 (M) 8 6.9 (M) 6.0 (M) 8.1 (H) 
Sub-Component 5: Marzano’s 
Vocabulary Cohort 2 15 7.1 (M) 7 6.9 (M) 6.4 (M) 8.4 (H) 
Total 31 7.1 (M) oll 6.9 (M) 6.0 (M) 8.4 (H) 
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Figure 4: Results of Year 3 Implementation Fidelity Scales 
Component |: Blended Intervention 
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The blended intervention model was implemented at a medium level of fidelity, with higher 
ratings for reading comprehension strategies and small-group instruction, and almost half of the 
schools from each cohort rated at high implementation for comprehension. 


Implementation of the blended intervention model for all students occurred at a medium level of 
implementation (7.2 out of 10), on average. Implementation of the focus on teaching reading 
comprehension strategies and the use of small-group instruction fell just shy of “high implementation” 
across schools, while average ratings for teaching academic vocabulary, use of the gradual release of 
responsibility model, and supporting differentiated instruction fell into the medium level of fidelity, with 
scores of 7.1, 6.8 and 6.8, respectively, on a 10-point scale. While some schools implemented the 
blended intervention with fidelity scores below 7.0, a few demonstrated a high level of implementation 
of this component, and almost half of the schools from each cohort (14 out of 31 in total) were rated at 
high implementation for the focus on reading comprehension (see Table F-2, Appendix F). 
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Teaching comprehension strategies.47 Teachers in the case study schools generally 
placed a high priority on teaching comprehension strategies, recognizing that the process helps 
students develop critical thinking skills. Evidence has begun to emerge of the benefits of longer- 
term program participation supporting the gradual release model, as was seen in case study 
schools from students in higher grades who have begun to use these comprehension strategies 
spontaneously during their reading. As one teacher noted, “[Striving Readers students] definitely 
come in knowing the strategies and you definitely can tell what students have been here since 
sixth grade, and what students have just transferred in—just because of what we do here, you 
know, the idea of knowing these strategies, knowing how to apply them, even when the teacher’s 


not telling you to apply them.” 


Observed lessons made use of the full portfolio of comprehension strategies, but surveys 
showed that questioning, predicting and inferring were taught most commonly, while text 
structure and metacognition were used least frequently. In observations, strategies were more 
often taught through explanation and direct instruction than through modeling, and most often 
were selected by the teacher rather than the students. 


Activities designed to teach comprehension strategies were observed in the large majority of case 
study observations, including those of both sixth and seventh grade ELA lessons, as well as many social 
studies and science lessons. According to the district-wide teacher surveys, the strategies that literacy 
teachers taught most commonly included questioning, predicting, and inferring, each of which were 
taught on at least a weekly basis by at least 90% of respondents, and almost daily by at least 37% of 
respondents. (It should be noted however that in quite a few case study observations, the lessons 
involved comprehension strategies applied to vocabulary review but not to extended reading). In case 
study observations, the use of comprehension strategies was built into many of the Striving Readers 
instructional methods. For example, as part of the PRC2 process, teachers might model the use of text 
structure by showing how to preview the table of contents, glossary, index, headings, and internal 
structure of a text. During the partner reading, students practiced questioning by writing down a 
question they had about the selection their partner had just read. Discussions about these questions with 
their partners often involved summarizing and making predictions and inferences about the text. 
Teachers found that this “portfolio” of comprehension strategies can be very useful for differentiating 
instruction to varied learning styles, as different students “understand in different ways ...because one 
student wants to make a prediction, [while]...it’s better for another one to ask a question.” During 
interviews, case study teachers also noted that the supportive literacy materials (both print and audio) 
provided through Striving Readers helped make instruction in comprehension strategies more effective. 


47 Findings about teaching comprehension strategies that are discussed here apply to the targeted intervention model 
(Table 14, sub-component 7) as well as the blended intervention model (Table 12, sub-component 3), which ideally 
should be coordinated with each other. The teaching of comprehension strategies during AMP classes is addressed in 
the discussion of the intensive intervention below. 
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During observed lessons, comprehension strategies were more often taught through explanation and 
direct instruction than through modeling, however, and in most cases it appeared that the strategy or 
strategies being taught were selected by the teacher rather than the students. Also, program-wide, some 
specific strategies were used less frequently than others, as evidenced by teacher surveys. Most notably, 
while the large majority of literacy teachers (89% or more) reported providing explicit instruction in most 
other comprehension strategies at least once a week, only three fourths (76%) and just over two thirds 
(71%), respectively, provided as frequent instruction in the use of text structure and metacognition for 
comprehension. 


It is not clear whether the smaller focus on these strategies was related to teachers believing that 
other strategies were more useful to students given their current reading skills, or whether it was related 
to the teachers’ own comfort levels for conceptualizing and communicating the strategy. The latter 
explanation seems most plausible in regard to metacognition, which was the most frequently cited (by 
59% of literacy teachers program-wide) of all Striving Readers techniques and strategies for which 
teachers would like to receive more training. At one case study school, the principal reported that 
instruction in comprehension strategies was taking place even before the program began; but several 
teachers at other case study schools commented that they needed more targeted training to help clarify 
the concepts and support their instruction in the comprehension strategies in general. 


Small-group instruction. 


Small-group instruction was being used with increasing frequency throughout the program— 
often within the context of a whole-part-whole framework—and was among the higher rated 
program components. Nevertheless, it was not always clear that the small group activity fully 
supported differentiated instruction. 


Small-group instruction within the whole-school model also received one of the higher ratings 
among the sub-components of the blended intervention. This finding is consistent with the program 
leadership’s observation that Striving Readers schools have been reconstructing how instruction is 
provided and that grouping is happening a lot more frequently—a condition that greatly helps to pave 
the way for differentiated instruction. One third (10 out of 31) of the treatment schools attained fidelity 
scores at the high level of implementation in this area, indicating that they used small-group, paired, 
and/or individual instruction on a regular basis, and specifically used Striving Readers materials such as 
the listening centers and classroom libraries to support these grouping strategies. This finding was very 
consistent with the case study observations, during which small-group, paired, and/or individual 
activities were used during almost every observed class. 


These small-group activities were frequently conducted within the context of a whole-part-whole 
framework. During case study observations, this structure was seen in use during the substantial 
majority of observed classes. Typical examples included a lesson starting out with the class working 
together as a whole, reviewing vocabulary, previewing a story, and/or receiving instructions or modeling 
from the teacher on how to approach the small-group activities. The classes would then separate into 
groups of pairs for small-group activities (examples of which are provided in more detail in the next 
section), and ultimately reconvene to discuss the activities with the whole class. In one well-designed use 
of this structure, for example, the whole class discussed the main character of a story they had read in 
order to discuss external features of the characters. The teacher then modeled the process of creating 
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her own fictional character by deciding what characteristics the character would possess. Students then 
worked in pairs to develop their own characters for a short story. Finally, the whole class discussed how 
an author can use characters’ internal qualities to influence their external look. Additional work on 
character development was planned for the following day. 


Examples from case study observations of where this structure was implemented well included 
classes that received clear guidelines, modeling of the assignment, or both, and where students moved 
seamlessly between whole-class and small-group activities, clearly demonstrating familiarity and comfort 
with the process. Ideally, the small-group portion of the structure should have been used to facilitate 
differentiated instruction; however, the extent to which this took place was not always clear, as discussed 
below. In addition, there were several observed lessons in which the structure did not close with a 
regrouping of the whole class. This may have been due to time constraints, and it was apparent that in at 
least some of these cases, there were plans to complete the whole-class regrouping on the following day. 
Indeed, this approach was seen in some classes that were observed for two consecutive days. 


Differentiated instruction. 


Among schools, fidelity ratings in differentiated instruction ranged from low to high levels of 
implementation, although buy-in from school administrators and teachers, as well as 
acceptance by students, appeared to be increasing. Collaboration with the LIT was considered 
an important condition, as was strong classroom management skills, availability of leveled 
materials, actively involving the students and supportive feedback. 


Less consistent than the use of small-group (and whole-part-whole) structures was the use of 
Striving Readers techniques, frameworks, materials, and technologies to support differentiated 
instruction. Several schools (four out of 31) did receive fidelity scores reflecting high implementation for 
this component, but program schools overall were rated at the middle level of implementation, and this 
was one of only two areas within the whole-school model (the other was gradual release) in which any 
school was rated at low implementation. 


Case study results indicate that, at least within these schools, buy-in to the concept of differentiated 
instruction continues to improve among teachers and administrators, who are encouraged by student 
successes observed through Striving Readers assessments such as reading benchmarks and fluency 
snapshots. Particularly given some teachers’ reluctance or discomfort around differentiation, support 
from the school administration was found to be an important factor in its success; in some cases the 
LITs noted that working with the school administration proved effective in encouraging some reluctant 
teachers. In this context, it is promising that principals acknowledged the need for additional support in 
differentiation. Equally important, students in case study schools have been found to be increasingly 
accepting of differentiated instruction, becoming less self-conscious about being separated into different 
groups. Some teachers have attributed this improvement in attitudes to a shift in their own emphasis, in 
which learning objectives and the concept of “success” are defined differently for different students. In 
other cases, however, staff have noted that this self-consciousness remains a problem, exacerbated in 
part by the fact that the students are less familiar with the LIT than they are with the classroom teacher. 


In their efforts to strengthen differentiation for all students, case study teachers have remarked on 
the importance of being able to work closely with the LIT, who is often more highly trained and 
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experienced in using this technique. Strong classroom management skills also have been found to be 
important in facilitating successful differentiation. Case study teachers have noted that differentiation is 
challenged by the difficulty of getting some students to focus; and the management skills needed to 
accomplish this are further challenged by the large class sizes found in many Chicago schools. In case 
study interviews, teachers have lauded the value of leveled literacy materials provided through Striving 
Readers to support differentiation (and observers confirmed the rich variety of print materials available 
in the classroom library of virtually every observed classroom); and the process is further facilitated by 
the availability of program-developed materials such as word-study and text sets that are explicitly 
structured to accommodate differentiation. However, some teachers have expressed concern that there is 
not always a sufficient quantity of any given text to have all students at a given level working on the same 
book. Access to a variety of assessment methods and to a varied instructional tool kit are also critical to 
support differentiated instruction, as they enable teachers and LIT's to identify and address students’ 
needs within specific skill areas. Increasing sophistication with differentiation has been observed among 
case study teachers who recognize the importance of their familiarity with and consideration of students’ 
personalities and interpersonal relationships as additional factors in forming groups. 


While program leaders feel there is still much progress to be made, true differentiation appears to be 
happening more frequently at case study schools, and a number of examples of lessons that clearly and 
explicitly used differentiation techniques were seen during case study observations. At the most basic 
level, the teacher, the LIT, or, most often, both, circulated among the groups to assess students’ 
understanding through observation and probing, and provided support targeted to particular groups or 
individuals as needed. This active involvement from the instructors seemed essential to the process, as 
students cannot always be counted on to seek out help on their own; indeed, spontaneous requests for 
help were seen only infrequently during observed lessons, and even collaboration with their peers often 
depended on teacher guidance to keep students focused. (Feedback from the instructor could prove 
counterproductive, however, if it was not delivered with encouragement. In one case in which the 
teacher’s comments were sharp and critical, for example, the students became withdrawn and reticent 
about interacting either with the teacher or among themselves.) Other clear examples of differentiation 
during observed lessons included groups organized by level and using distinct methods to develop 
vocabulary—studying vocabulary sheets, looking up words in a dictionary, or discussing metaphors in 
haiku; a class in which each learning station targeted specific skills—fluency, word knowledge, or 
comprehension; and small groups working on the same activity but each allowed to progress at its own 
pace. 


Expanding the use of differentiated instruction is challenged by factors such as traditional 
teaching habits, time constraints, large class sizes, discipline problems, lack of timely access to 
assessment results (which were more often used for “grouping” than for “differentiating 
instruction’) and, in some cases, limited opportunity to meet with the LIT. Some 
misunderstandings about the technique were evident from the infrequent use of modeling and 
rotation of learning stations among the whole class. 


While the above successes are notable, there remain substantial challenges, as well as ambiguities 
about how differentiation is being understood and applied; and the project director and senior literacy 
advisor, as well as many principals, have acknowledged these challenges. Traditionally, middle school 
teachers are accustomed to teaching to the whole group, and some case study teachers continue to push 
back against the use of leveled groupings. Case study observations and teacher interviews revealed that 
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time constraints and large classes interfere with instructors’ ability to provide individualized feedback, 
and discipline problems make it difficult to get struggling students to focus. In many cases, even where 
the groups were working on distinct activities at different learning stations, the groups rotated during the 
course of the lesson, making it less clear whether the different activities were really achieving 
differentiation. 


Intervention is also intended to be supported through the use of scaffolding techniques (such as 
expanding on students’ ideas, asking probing questions to prompt students to deepen their thinking, and 
providing initial modeling of activities) to help struggling readers catch up. During case study 
observations, discussion and expansion of ideas and targeted probing were common occurrences; 
however, modeling was seen much less often than providing instructions about the activity followed by 
class discussion to ensure understanding. While this approach seemed to be effective for many students, 
the apparent infrequency of modeling may put students with certain learning styles at a disadvantage. 


Teachers indicated that time constraints present an ongoing challenge, interfering with their 
opportunities to explore and interpret achievement data; develop leveled activities in learning centers; 
work directly with all student groups (especially in larger classes); and collaborate with the LIT. Indeed, 
while the vast majority of surveyed ELA and self-contained teachers district-wide reported that they met 
with their LIT informally or through team meetings (85% and 82%, respectively) at least once a month, 
only about half (55%) were able to meet individually on at least a weekly basis (see Table 14). 


Table 14: Spring 2009 Literacy Improvement Survey: Literacy Teachers 
How often do you meet or collaborate with the LIT in the following settings? 


Less than 1-3 
once a times a 1-3 times) | 4-5 times 
Never month leateyarsal a week a week 


Impromptu one-on-one meetings (during 
lunch, prep periods, before/after school, etc.) 


12.4% | 7.9% 24.9% 35.6% 19.2% 


Literacy (vertical and horizontal) team 


; 175 
meetings 


12.6% | 5.7% 56.6% 22.3% 2.9% 


Given some case study teachers’ concerns that too much time during team meetings is devoted to 
“administrative” issues (which they distinguish from planning time), the additional time that they spend 
with LITs during such meetings may not be providing as much value added as it could. In addition, 
while some teachers continue to express a need for additional training in differentiation—particularly 
(though not exclusively) as it applies to English language learners—others feel that redundancy in the 
professional development program is exacerbating time constraints and that the time could be better 
used for analysis of assessment results. 


Case study teachers have also expressed concerns that lack of timely access to assessment data can 
interfere with their ability to identify student needs and plan differentiated instruction. Indeed, even 
among LIT, fewer than half of the respondents to the LIT survey (45%) reported that student 
assessment data were used for differentiating instruction “to a large extent.” 
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Concerns have also been raised by project leadership that teachers often confound the concepts of 
differentiation with small-group instruction, and to the extent that some teachers’ or administrators’ 
understanding of differentiation may be incomplete, the fidelity ratings based on surveys of these staff 
may be inflated. Unfortunately it was often difficult for an interviewer to infer what a case study 
respondent meant by their use of the term, and even during “differentiated” activities in observed 
classrooms, it wasn’t always possible to know whether each activity was specifically targeted to the needs 
and strengths of the students in each group. It is notable, however, that, even though many of the 
observed small-group activities were reported by the instructor to have been formed by ability level, 
there were many cases where all groups were working from the same materials and engaged in the same 
assignment. (This was true even more often during lessons in the content areas, where the class was all 
working from the same textbook.) The fact that, program-wide, a larger proportion of LIT's reported 
using assessment data for creating in-class instructional groups than for differentiating instruction (65% 
vs. 45% of survey respondents, respectively, reported doing each “to a large extent’) seems consistent 
with the possibility that LITs, at least in some cases, are recognizing the distinction but are perhaps not 
fully prepared to implement differentiation. 


Instructional frameworks and techniques. 


The PRC2 framework was used by the large majority of literacy teachers, but it was not used as 
regularly as expected. Teachers used it to help develop comprehension, fluency and vocabulary, 
and also as a vehicle for differentiated instruction. In some cases, however, application of the 
technique was less student-focused than the model intends. 


The PRC2 framework was used by the large majority of literacy teachers in the Striving Readers 
schools to support small-group activity and differentiated instruction: over 90% of these teachers 
program-wide reported that they used it at least monthly to help develop comprehension; and 84% and 
78%, respectively used it at least monthly to develop fluency and vocabulary. (However, many teachers 
did not use this framework as regularly as expected—only 50% to 59% did so at least weekly for each of 
these purposes, and nontrivial numbers—9% and 7%, respectively, program-wide—said they never used 
it to develop vocabulary or fluency.) As one of the most frequently employed Striving Readers 
techniques, PRC2 was widely considered to be successful by the majority of case study interviewees. 
During case study focus groups, teachers noted that PRC2 was valuable for enabling students to work 
with others who share their fluency level, building their ability to have independent, thoughtful 
conversations about literature, and increasing their interest in reading nonfiction texts. 


Among case study schools, many classes were observed in which there were clearly established PRC2 
routines that students followed readily, showing engagement in their readings and active collaboration 
with their partners. Within the PRC2 activities, differentiation was accomplished by assigning partners 
to activities addressing specific skills (fluency, word patterns, comprehension, etc.) and/or to work with 
informational materials targeted to each pair’s independent reading level. Teachers or LITs provided 
scaffolding to partners having difficulty clarifying their questions, but in some cases, student pairs 
responded to questions on a teacher-prepared work sheet rather than discussing their own questions. 
While a few lessons were observed during which pairs exchanged ideas from different texts during the 
whole-class discussion, in many cases all pairs worked from the same text or even the same passage— 
although differentiation can still occur in these circumstances to the extent that the LIT and/or 
classroom teacher provide individualized feedback and scaffolding to partners as they are working. 
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Most schools implemented a systematic approach to teaching academic content vocabulary 
at a medium level of fidelity, using methods from Marzano’s Vocabulary, but not as frequently 
as expected. 


On fidelity ratings reflecting the implementation of a systematic approach to teaching academic 
content vocabulary for the whole class, five schools attained a high level of fidelity. High ratings in this 
area reflect frequent use of and comfort with instructional methods that are built in to Marzano’s 
Vocabulary, such as explicit instruction; modeling word parts; vocabulary notebooks; and before, during, 
and after reading strategies (although the ratings do not necessarily indicate that teachers were using 
Marzano’s Vocabulary per se). The remaining schools implemented at a medium fidelity level, indicating 
that they used these practices to build students’ vocabulary, but used each one less than once a week, on 
average. Observed lessons at case study schools that did provide vocabulary instruction built it in to 
activities such as reading articles on content area topics, deriving definitions from a story, or reviewing 
vocabulary lists with each word used in a sentence provided by the teacher. Many of these lessons 
focused on vocabulary relevant to the content areas, such as a social studies lesson in which students 
used Marzano’s Vocabulary as part of a study of inventions, and a science lesson in which students 
reviewed vocabulary related to botany. 


Large class sizes and time constraints made it difficult to implement a wide variety of 
Striving Readers techniques; but teachers might address these challenges, while also improving 
differentiation, by applying techniques mote selectively. Some teachers also perceived that 
PRC2, or the program in general, is less well-suited proficient readers than to struggling readers. 


Implementation of these instructional techniques at the whole-school level encountered several 
challenges as well. Although not specific to Striving Readers, large class sizes make it more difficult to 
implement many of these techniques, and case study teachers expressed concern that there is often 
insufficient time during each class to implement a variety of techniques. Since there is no expectation in 
the program model that all students participate in all instructional techniques, this concern may indicate a 
continued need for teachers to further strengthen their skills in applying techniques selectively and, 
perhaps, concurrently, as a means of differentiating instruction. As discussed above in relation to 
differentiated instruction, some case study teachers also expressed concern that they do not have a 
sufficient quantity of certain materials, which can be an impediment to implementation for a variety of 
instructional activities, depending on the materials. (However, this may be another indication that 
teachers are resorting to having the whole class work on the same activities or with the same materials, 
rather than assigning activities and materials according to need.) 


Some case study teachers have also indicated that they feel that the PRC2 framework is targeted to 
struggling readers, but is not appropriate for the whole classroom because the books are leveled below 
grade level. Indeed, related comments were made that the program as a whole is conceptualized more 
for supporting struggling readers, and the techniques and materials do not adequately challenge other 
students. Program leadership contends, however, that Striving Readers materials are leveled 
appropriately for more proficient readers as well. This perception among some teachers may therefore 
indicate a need for further training. 


68 


Component 2: Targeted Intervention 


Fidelity scale results for the targeted intervention are presented in Table 15 and Figure 5. 


Table 15: Results of Year 3 Classroom Model Implementation Fidelity Scales: 
Targeted intervention 


Fidelity Component Cohort | N 


Component 2: 


Targeted Intervention 


Sub-Component 6: Teacher/LIT 
collaboration 


Sub-Component 7: Small-group 
setting for Tier 2-3 students with 
direct instruction in 
comprehension, vocabulary, and 
fluency 


Figure 5: Results of Year 3 Implementation Fidelity Scales 
Component 2: Targeted Intervention 
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Cohort | 16 8.2 (H) 8 8.3 (H) 6.6 (M) 9.5 (H) 
Cohort 2 15 8.1 (H) 1.0 8.1 (H) 5.5 (M) 9.4 (H) 
Total 31 8.1 (H) 9 8.1 (H) 5.5 (M) 9.5 (H) 
Cohort | 16 8.0 (H) 1.1 7.9 (M) 5.9 (M) 9.5 (H) 
Cohort 2 15 8.1 (H) 3 8.3 (H) 4.4 (L) 9.6 (H) 
Total 31 8.1 (H) ll 22 8.3 (H) 4.4 (L) 9.6 (H) 
Cohort | 16 8.4 (H) 1.0 8.6 (H) 6.0 (M) 9.8 (H) 
Cohort 2 15 8.0 (H) II 8.1 (H) 6.5 (M) 9.6 (H) 
Total 31 8.2 (H) 1.0 8.5 (H) 6.0 (M) 9.8 (H) 
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The targeted intervention model was implemented at a high level of fidelity, on average, 
although a larger number of Cohort 1 schools had high sub-ratings of direct instruction for Tier 
2 and 3 students, while a larger number of Cohort 2 schools had higher ratings for teacher/LIT 
collaboration. Teachers, principals and LITs agreed that the targeted interventions had helped 
to improve their struggling readers’ literacy skills, as has been verified in some cases by 
authentic assessments. 


Fidelity ratings for the targeted intervention model reflected the quality of targeted instruction for 
Tier 2 and 3 students from the LIT. Relevant aspects of implementation include the frequency, content, 
and effectiveness of the collaboration between literacy teachers and the LIT for the purpose of planning 
instruction for struggling readers (a part of their overall collaboration goals as discussed in the previous 
section), and the implementation of grouping strategies to provide direct and explicit instruction in 
comprehension strategies, vocabulary, and fluency to Tier 2 and 3 students. This model was 
implemented at a high level of fidelity, on average. Cohort 1 and Cohort 2 schools had average 
implementation ratings on this component of 8.2 and 8.1, respectively, although at the building level, a 
larger number of Cohort 1 than of Cohort 2 schools (12 out of 16, vs. 8 out of 15) had a high level of 
implementation (see Table F-3, Appendix F). This was also true of sub-ratings of direct instruction in 
comprehension, vocabulary, and fluency for Tier 2 and 3 students, in which 13 of 16 Cohort 1 schools 
and 8 of 15 Cohort 2 schools were rated as having high implementation. For the second sub- 
component—teacher/LIT collaboration for the purpose of planning targeted instruction—schools again 
achieved an average rating at a high level of fidelity; in this case, however, a larger number of Cohort 2 
schools than Cohort 1 schools were rated at high fidelity (11 out of 15, vs. 7 out of 16). One Cohort 2 
school, however, appears to have experienced difficulty with this collaboration, as evidenced by an 
implementation score at the low end of fidelity. 


According to the program-wide surveys, most literacy teachers believed that the efforts of the LIT 
had helped to improve their struggling readers’ literacy skills, with over three fifths (62%) judging them 
as effective, including one third (34%) reporting that they had been very effective. Their principals generally 
shared this appreciation for the role of the LITs. These positive assessments were reflected by the 
comments of some LITs in the case study schools, who were confident that the targeted interventions 
have shown successes, demonstrating the potential for the process to increase student confidence and 
engagement. As one LIT noted, “There’s more student engagement because you’re able to put them in 
small groups, or you’re able to give them the one-on-one attention that they need.” The comments of 
these LIT’s also indicated that there have been increases in concrete literacy performance as measured by 
the authentic assessments being promoted by the program. 


While the majority of teachers and principals clearly valued theit LITs, a troubling number 
of teachers felt that their LITs had been ineffective. Challenges to targeted intervention 
included different pacing needs for struggling readers, time constraints aggravated by large 
class sizes and by LITs’ involvement in coaching for the whole class, and a sense that some Tier 
2 students need the intensive intervention. 


While the above findings about the perceived value of the LITs are very encouraging, it is of concern 


that one in five classroom literacy teachers responding to the survey (21%) felt that their LIT had been 
minimally effective or ineffective in these efforts. 
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A number of challenges have been observed that hamper the targeted intervention process. Pacing 
can present significant difficulty, as the LIT strives to keep up with the rest of the class, even if the 
targeted students may still need continued work. As one LIT observed during the case study, “The 
challenge is to be able to work with them consistently, every day, because they move so fast. You’re 
never quite sure if they have mastered or if the intervention has served its purpose, because oftentimes 
the class constantly moves on so you don’t want to hold them back from not moving on with the class. 
There’s always a constant infusion of new material without you having definite assurance that they’ve 
mastered what you just taught them. You have to move on, and then you offer the assistance on the 
new lesson.” The LITs’ additional role of providing the classroom teacher with coaching assistance for 
all students puts an additional strain on their time and adds to this challenge. Given the belief that has 
been expressed by program staff at all levels since the Tier 3 cutoff was lowered, that some Tier 2 
students require the support of the after-school program, the difficulty of keeping struggling readers 
apace with the rest of the class is even greater. Furthermore, the large class sizes mean larger numbers of 
Tier 2 students receiving targeted intervention services from the LIT. 


Teacher/LIT planning for struggling readers. 


Staff clearly valued the opportunities for teacher/LIT collaboration during literacy team 
meetings. Grade level meetings, however, were often perceived as less productive due to their 
much broader focus. 


Collaboration between LITs and classroom teachers can occur in a variety of settings, including 
formal meetings such as literacy team meetings and “horizontal” (grade-level) meetings, as well as 
informal, one-on-one meetings during staff planning periods or free time. During interviews and focus 
groups at the case study schools, staff clearly indicated that they valued the literacy team meetings, 
consistently describing them as opportunities for ideas and classroom practices to be shared and 
developed. They described using these meetings to identify best practices and plan new strategies for 
implementation in the classroom. 


At smaller schools where there are few teachers at a given grade, grade-level team meetings often do 
not occur, and formal planning meetings for the Striving Readers program occur primarily through the 
literacy team. Where they do occur, however, many case study teachers have spoken less favorably about 
the value of grade-level meetings for program planning. In theory at least, the structure provides a 
unique opportunity for teachers and LITs to discuss the application of literacy instructional techniques 
and differentiation in other content areas, and this sharing does take place to some extent. However, 
quite a few teachers spoke about various obstacles to effective collaboration during these meetings. 

Time constraints are a challenge for both grade-level and literacy meetings. In grade-level meetings, 
however, teachers (and at least one principal) note that these constraints are exacerbated by the much 
broader focus of such meetings. Some case study schools have learned to use the meeting time more 
productively by asking team leaders to come prepared with specific agendas and a “vision” of what they 
want to accomplish in the hour. At another school, the principal makes a point of ensuring that there 
are 20 minutes of highly structured time dedicated to discussing literacy instruction during each meeting, 
These solutions were more the exception than the rule, however, even among other case study schools, 
where teachers were frustrated that the meetings were less productive because of the broad focus. These 
teachers felt that there just was not enough time to address all the topics they should, address everyone’s 
professional development needs, and focus on lesson planning rather than just “administrative issues.” 
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As the seventh-grade teachers at one school said, “It’s just too distracting, there are way too many 


people, too many conversations going on...and you feel as though you have to explain to everybody else 


what you’re talking about.” 


Component 3: Intensive Intervention 


Fidelity scale results for the intensive intervention are presented in Table 16 and Figure 6. 


Table 16: Results of Year 3 Classroom Model Implementation Fidelity Scales: 
Intensive intervention 


Fidelity Component Cohort | N Mean Score Nelaree ge] Median Minimum Maximum 
Deviation Score Score Neto] a= 

Cohort! | 16 | 8.2(H) 6 8.2 (H) 7.3 (M) 9.1 (H) 

Component 3: 
. Cohort2 | 15 | 8.2(H) 5 8.3 (H) 6.8 (M) 9.0 (H) 

Intensive Intervention™! 

Total 31 | 8.2(H) & 8.2 (H) 6.8 (M) 9.1 (H) 

Cohort! | 16 | 61(M) II 5.8 (M) 43 (L) 8.3 (H) 
aca al tance Cohort2 | 15 | 5.4(M) 1.2 5.1 (M) 2.8 (L) 7.6 (M) 
instructional time 

Total 31 | 5.8(™) 12 5.6 (M) 2.8 (L) 8.3 (H) 

Cohort! | 16 | 9.6(H) 9 10.0(H) | 6.8(M) 10.0 (H) 
oe a aomallgrouP.. Genome | 15 | 100d) 0 10.0 (H) 10.0 (H) 10.0 (H) 

Total 31 | 9.8(H) a 10.0(H) | 6.8(M) 10.0 (H) 

Cohort! | 16 | 10.0(H) | .0 10.0(H) | 10.0(H) | :10.0(H) 
sub; Component IiDirec Cohort2 | 15 | 10.0(H) | .0 10.0 (H) 10.0 (H) 10.0 (H) 
instruction in comprehension 

Total 31 | 10.0(H) | 0 10.0(H) | 10.0(H) | :10.0(H) 

Cohort! | 16 | 7.7(M) 12 7.5 (M) 5.3 (M) 10.0 (H) 
eee Iniciar cali Cohort2 | 15 | 7.5(M) 9 7.5 (M) 5.9 (M) 8.8 (H) 
instruction in vocabulary 

Toul 31 | 7.6 (M) II 7.5 (M) 5.3 (M) 10.0 (H) 

Cohort! | 16 | 7.7(M) 1.6 8.1 (H) 4.0 (L) 9.6 (H) 
subs Somponens I2ePirect Cohort2 | 15 | 7.8(M) 1.6 8.0 (H) 3.5 (L) 10.0 (H) 
instruction in fluency 

Total 31 | 7.8 (M) 6 8.0 (H) 3.5 (L) 10.0 (H) 


lal Students who were not enrolled in AMP were scored as having 0% attendance. 


Te 


Figure 6: Results of Year 3 Implementation Fidelity Scales 
Component 3: Intensive Intervention 
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Low 0% 0% 0% 


AMP implementation was rated at a high level of fidelity across all components, and across 
all schools in both cohorts. Instruction in comprehension strategies was provided several times 
a week, and almost all programs maintained a low student:teacher ratio. 


Implementation of the AMP program was rated at a high level of overall fidelity across all 
components and schools, with ratings for Cohort 1 and Cohort 2 schools both averaging 8.2 out of 10. 
Most notably, all 31 treatment schools had perfect scores for explicit instruction in comprehension strategies, 
indicating that they provided such instruction in at least one of the seven comprehension strategies 
several times a week. (It was not expected that they provide instruction in each strategy at any particular 
frequency, since the strategies that a class focuses on depend on the students’ specific needs.) In 
addition, the schools were very successful, overall, in maintaining small-group settings with 
student:teacher ratios of 15:1 or less; only three schools (all Cohort 1) had classes above this ratio, and 
only two exceeded this class size by more than two students (with a total of 19 and 22 students enrolled, 
respectively). Explicit instruction in vocabulary and in fluency occurred at a medium level of fidelity, 
averaging 7.7 and 7.5, respectively, for Cohort 1 schools, and 7.7 and 7.8 for Cohort 2. Three schools 
(two from Cohort 1 and one from Cohort 2) had low fidelity in fluency. Overall, however, 
implementation of the AMP classes was judged as quite successful. 


Increased instructional time. 


The major challenge for the intensive intervention was providing increased instructional time to 
all eligible students. Classes started late due to delays in release of ISAT data and tier 
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assignments, and ended a week early due to other scheduling needs. More significantly, only 
three-fifths of eligible Tier 3 students enrolled in the class (with enrollment rates even lower at 
Cohort 1 schools). Factors contributing to low enrollment included access to transportation; 
safety concerns; competition with other academic and extra-curricular after-school programs; 
and, in substantial numbers of cases, perceptions that program content was not motivating, 
appealing, and/or appropriate to the reading levels or learning styles of all participants. In 
addition, attendance varied widely among those who did enroll. 


A much gteater problem for the intensive intervention was the extent to which schools succeeded in 
providing increased instructional time for all eligible students. According to the model, intensive 
intervention should be provided for at least four hours per week on average, and program leadership 
indicate that AMP classes should meet from early October through the first week of May. While most, if 
not all, schools did hold AMP classes for four hours per week,*8 classes were not able to start until the 
third week in October due to delays in availability of ISAT data, which held up the tier assignment 
process. In addition, all AMP classes ran only through the end of April, in part because of the need for 
schools to align their AMP schedule with other after-school programs. 


More serious than these scheduling limitations, however, were enrollment levels and variations in 
students’ attendance rates. Among those who were enrolled, attendance varied widely. More 
importantly, 42% of all sixth-graders who were assigned to Tier 3 at the beginning of the 2008-2009 
school year did not enroll in AMP. As a result, fidelity scores for increased instructional time were at the 
lower end of a medium level of fidelity, with a total of nine schools (of which seven were Cohort 2) 
receiving low fidelity ratings in this area. Even in spite of the shorter time frame, most schools were still 
able to offer an AMP schedule that provided almost as many hours as would have been in the full 26 
weeks; however, actual attendance averaged only about one-third (35% among Cohort 1 schools and 
32% among Cohort 2) of the total hours that students should have attended according to the model.*? 
The degree to which this resulted from non-enrollments (which were counted as 0% attendance) was 
greater for Cohort 1 schools, in which only 55% of Tier 3 students enrolled in AMP, than for Cohort 2 
schools, in which 67% enrolled. 


The project director, senior literacy advisor, and many principals have expressed awareness that they 
ate not serving all Tier 3 students, and have agreed that lowered participation rates have a lot to do with 
the fact that AMP is provided through an after-school program. Among the factors that have been cited 
as interfering with enrollment in after-school programs are access to transportation options, safety issues 
in some neighborhoods related to staying late, and competition with other after-school programs— 
including the federal Supplemental Educational Services (SES) programs that provide academic support 
at many schools, as well as extracurricular programs in sports, music, and art. As collaboration between 
the district team and schools continues to improve, project leadership report that some principals have 
made efforts to improve AMP attendance, including offering child care to younger siblings, providing 


48 Several schools reported attendance schedules only in days rather than hours; however, program leadership reported 
that all schools met for a total of four hours per week regardless of the number of days. 


” The dramatic change from past years’ attendance rates probably reflected the fact that previous attendance rates did 
not account for non-enrollment. 
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attendance incentives, and calling parents directly. In other cases, however, the project director feels that 
they need to get other principals to take more ownership in the program. 


Enrollment is ultimately the parents’ decision, however, and quite a few opt out. This decision may 
be related to the factors discussed above in many cases (parents are not required to provide a reason for 
opting out), but even AMP instructors have acknowledged that the program may not be ideal for all 
students. In response to spring surveys, the vast majority of AMP instructors (95%) felt that the 
program was at least “somewhat” appropriate (including 44% who felt it was “very appropriate’’) to the 
reading levels of the students who were participating; and majorities (68% and 61%, respectively) also 
felt that the program was appropriate to the literacy needs and learning styles of “most” participating 
students. However, similar proportions (61% each) felt that the program is “motivating” and “relevant 
to [the] interests” of only half of participants or fewer (see Table 17). 


Table 17: Spring 2009 Literacy Improvement Survey: 

LITs and Other AMP Instructors 

Proportion of Students for Whom You Think the Following Statements About 
AMP Are True 


All or Hardly 
: Most A few 
The AMP after-school program is... almost all any 
students students 

students students 
... engaging. 4l 19.5% 36.6% 26.8% 12.2% 4.9% 
...relevant to their interests. 4l 14.6% 24.4% 41.5% 17.1% 2.4% 
... motivating. 4l 12.2% 26.8% 39.0% 12.2% 9.8% 
..-appropriate to their literacy needs. 4l 24.4% 43.9% 17.1% 12.2% 2.4% 
...appropriate to their learning style. 4l 19.5% 41.5% 17.1% 12.2% 9.8% 


It is not clear whether parents share this impression nor whether this may be a factor in some 
parents’ decision to opt out. Nevertheless, the question remains whether AMP is effective for those 
students who do attend, even among those for whom it may not be as engaging as it could. This 
question is addressed under Section V below. 


Direct instruction in comprehension, vocabulary, and fluency in a small-group 
setting. 


Observed AMP implementation at the case study schools was generally successful, with well- 
planned activities, established routines, use of grouping structures, focus on comprehension 
and vocabulary, and differentiation through use of concurrent instructional methods and 
individualized support from the instructor. 


The perceived appropriateness of the AMP program as discussed above necessarily reflects the 
appropriateness of the program as it is being implemented, and as it is understood by responding staff, 


5° Includes all 31 LITs, plus 10 classroom teachers teaching AMP at grades 7 or 8. 


75 


not necessarily the appropriateness of the intended model. Given that fidelity ratings, while very high in 
comprehension instruction, were somewhat less positive—and in a few cases, problematic—in 
vocabulary and fluency instruction, it is possible that this distinction is an important one. 


The 10 AMP classes that were observed at the case study schools, however, provided generally 
positive examples of AMP implementation. Activities seemed well planned and orchestrated and 
students were clearly involved in and accustomed to the various instructional techniques. The framework 
for differentiated instruction was apparent through varied grouping structures that included whole-class, 
small-group, and individual activities as needed. Examples of further refinement of instruction were also 
observed through mid-course adjustments to activities. For example, when a group of students indicated 
that the vocabulary in an activity was too difficult, the LIT revised the lesson to do the activity with the 
whole class. At the same time, differentiation was still maintained through the instructor’s 
individualized, targeted questioning and support for vocabulary and pronunciation. While differentiation 
may have been achieved even mote effectively by providing different activities to groups at different 


levels, this example demonstrates that differentiation still can be achieved even when planning time is 
limited. 


In other observed AMP classes, efforts to provide differentiation through varied activities were more 
apparent. These classes frequently involved a variety of activities happening at any given time, such as 
small-group instruction with teacher guidance, independent reading, partner reading, or completing AMP 
workbook activities. Observed classes frequently included substantial focus on comprehension 
strategies, particularly predicting, summarizing, questioning, and visualization, often within the context of 
partner or independent reading; as well as other instructional techniques, including Marzano’s 
Vocabulary and KWL. 


e Mote generally, case study observations of AMP classes provided several examples of 
appropriate activities, including the following: 
e Use of a variety of targeted, concurrent instructional methods, activities, and tools while the LIT 


circulated and provided guidance and feedback. 


e Appropriate applications of technology that incorporated student-student and student-teacher 
interactions. Applications included using the handheld computers for KWL, to complete 
graphic organizers on PicoMap for vocabulary study, and to create drawings in Sketchy for 
visualization; conducting partner reading using materials from the Internet; and using Playaways 
for fluency practice. 


e Lessons included a clear focus on vocabulary with embedded writing activities, using real-life 
analogies for scaffolding. 


e Use of AMP workbooks individually and in pairs. 


During these observations, most students were fully engaged, focused, and on task, while teachers 
worked to draw in quiet students and gently guide them to on-task activities. 
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Component 4: Purposeful Assessment and Data-Driven Instruction 


Fidelity scale results for purposeful assessment and data-driven instruction are presented in Table 18 and 


Figure 7. 


Table 18: Results of Year 3 Classroom Model Implementation Fidelity Scales: 
Purposeful Assessment and Sateuriven Instruction 


Fidelity Component Cohort Mean Score Standard Median Minimum Maximum 
Deviation Score Score Score 
Component 4: Cohort | 7.7 (M) 8.1 (H) 5.3 (M) 8.9 (H) 
Purposeful Assessment & Data- | Cohort2 | 15 | 6.8 (M) 1.2 6.9 (M) 44 (L) 8.4 (H) 
Driven Instruction 
Total 31 7.3 (M) 1.2 7.6 (M) 4.4 (L) 8.9 (H) 
Cohort | 16 7.2 (M) 1.0 7.4 (M) 4.8 (L) 8.6 (H) 
Sub-Component 13: Whole- 
scibol (blended) interyenaon Cohort 2 15 6.9 (M) 1.0 7.1 (M) 4.8 (L) 8.1 (H) 
Total 31 7.0 (M) 1.0 7.3 (M) 4.8 (L) 8.6 (H) 
Cohort | 16 8.3 (H) 1.8 9.2 (H) 5.0 (L) 10.0 (H) 
ee ene payee Cohort2 | 15 | 6.7(M) 2.1 6.7 (M) 3.3 (L) 10.0 (H) 
intervention 
Total 31 7.5 (M) Dal 7.5 (M) 3.3 (L) 10.0 (H) 


re 


Figure 7: Results of Year 3 Implementation Fidelity Scales 
Component 4: Purposeful Assessment and Data-Driven Instruction 
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The implementation and use of authentic assessments was rated at a medium level of 
fidelity overall, with mean scores of 7.7 for Cohort 1 schools and 6.8 for Cohort 2. There were 
indications that the use of data was beginning to become part of the culture in Striving Readers 
schools, including recognition that a variety of assessments were needed to provide a full 
picture. This culture is enabling some schools to see progress in students’ literacy skills that 
may not be detected by the ISAT. 


However, assessment data were often used more to monitor individual progress than for 
broader lesson planning, and its importance for differentiated instruction was not always 
recognized. Other challenges to using assessments included the timing of the release of the 
data or the time needed for teachers or LITs to administer them (especially in large classes). 
However, spelling inventories and fluency snapshots have been cited as useful yardsticks that 
are easy to implement. 


The use of purposeful, authentic assessments and application of assessment results to data-driven 
instruction was rated at a medium level of fidelity overall, with an average score of 7.7 among Cohort 1 
schools, and 6.8 among Cohort 2. At the individual school level, 8 of 16 Cohort 1 schools were rated at 
high implementation in this area, compared with 3 of 15 Cohort 2 schools (and one Cohort 2 school that 
rated at low fidelity). These scores reflect the extent and quality with which principals said that 
assessment data was used in their schools for purposes such as screening, diagnosing, and monitoring 
student progress and planning differentiation and professional development. Scores also reflect the 
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extent to which classroom literacy teachers reported that assessment data were used for these purposes, 
as well as whether teachers used various specific authentic assessments for the purposes for which the 
model intended them.*! 


While significant challenges remained during Year 3 in the effective use of assessment for data- 
driven instruction, there were also significant indications that the use of data was beginning to become 
part of the culture in Striving Readers schools, even before this occurred citywide with the new district 
administration. Case study interviews revealed that teachers were coming to understand that various 
types of assessment data need to be considered in order to get the “whole picture of the child.” For 
example, these teachers pointed out to interviewers that fluency snapshots are not intended to provide a 
measure of comprehension, while the BRI is not intended to assess fluency, and that looking at either 
one in isolation would not provide as complete a picture of a student’s abilities. Using a variety of 
assessments that focus on different skills has also enabled teachers to recognize gains that their students 
are making, even though they may not be detected by the ISAT. As a result, teachers began focusing 
more not only on performance levels, but also on “celebrating growth.” These more nuanced 
understandings of student progress have also been used, in some cases, to foster self-reflection among 
students about their own progress, which has proven important to improving student motivation. In the 
best examples in the case study, assessment data were used, often through planning during literacy team 
meetings, throughout the school hierarchy—from teachers to LITs to assistant principals to principals— 
creating a common language so that the whole staff was on same page about student progress and areas 
of need. 


During district-wide interviews, however, principals often indicated that they recognized that many 
teachers were still not that good at using “real time” assessment data to inform instruction. While 
principals often seemed to understand the distinction between grouping strategies and true 
differentiation—as evidenced by comments such as one describing a school’s efforts to “determine 
[students’] needs in terms of reading standards, and trying to do small group intervention based on their 
academic deficits” —there were indications that classroom teachers were not always making this 
distinction. Often, data were being used more to determine individual students’ progress than for 
broader strategizing and lesson planning around classroom-wide trends. Some case study teachers 
described the time spent reviewing assessment data during literacy team meetings as a trade-off with 
planning differentiated instruction, which would seem to indicate that these teachers—and perhaps other 
literacy team members as well—may have been missing a fundamental point about the role that such 
data play in differentiated instruction. 


Time constraints and timing of assessments have also posed challenges to the optimal use of data. 
During the case study, both teachers and LITs expressed frustration about the time it took to receive 
certain assessment results. They also wished they had had access to more easily-implemented 
assessments that they could administer themselves as needed, so that results would be available on a 
timely basis to guide differentiated instruction. 


5! These scales included separate ratings that were specific to the use of assessments for planning the whole-school 
intervention and the intensive intervention; however, distinctions in the ratings between these two components were 
less reliable than the score as a whole, since the intensive intervention sub-score was based on only four items. 
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This concern applied to tier assignments as well, which typically are not completed until more than a 
month into the school year due to the timing of availability of ISAT data. During one teacher interview 
from the case study, frustration with this situation was palpable. As one teacher commented, “I would 
say that there is a huge glitch in the [tiering] system, because the kids didn’t get identified until, like, mid- 
October. So, what happens is, I suggested certain children for after-school tutoring, assuming that some 
of my lowest readers would be in the LIT’s after-school program, and they weren’t. ...[T]hey were 
changed. When they finally came in, remember, they were changed too. Some of the children were 
identified as Tier 3, and then they went to Tier 2.” These comments stand in contrast, however, to 
project leadership’s observation that, at least at the beginning of the year, it was difficult getting schools 
to make assessment a priority, and it took a long time for some LIT to complete the reading inventories 
that were critical for work with struggling readers. 


The need to utilize assessments that can be administered quickly emerged as an important 
consideration, given that most assessments are time intensive, and their administration is often 
constrained by the large class sizes. (This may have been part of the problem in completing the reading 
inventories in the fall.) At least some of the assessments that are already used in the program, 
however—particularly spelling inventories and fluency snapshots—have been cited by teachers and LITs 
as providing useful yardsticks while still being easy to implement. 


Component 5: Materials 


Fidelity scale results for integrating high-quality, high-interest materials are presented in Table 19 and 
Figure 8. 
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Table 19: Results of Year 3 Classroom Model Implementation Fidelity Scales: 


Materials 
Fidelity Component Cohort Mean Score Standard Median Minimum Maximum 
Deviation Score Score Nefol a= 
Cohort | 6.1 (M) 4.8 (L) 7.9 (M) 
Component 5: Materials"! Cohort 2 15 6.0 : 7 1.0 6.0 (M) 4.1 (L) 7.5 (M) 
Total 31 5.9 (M) 1.0 6.0 (M) 4.1 (L) 7.9 (M) 
Cohort | 13 9.0 (H) 1.6 10.0 (H) 6.7 (M) 10.0 (H) 
Sub-Component 15: Text sets"! Cohort 2 1 7.9 (M) a7 10.0 (H) 3.3 (L) 10.0 (H) 
Total 24 8.5 (H) 2.2 10.0 (H) 3.3 (L) 10.0 (H) 
Cohort | 16 5.9 (M) 2.3 6.2 (M) 1.4 (L) 8.7 (H) 
i be Sehoe! Cohort2 | 15 | 5.5 (M) 2.2 6.1 (M) 1.0 (L) 8.4 (H) 
Total 31 5.7 (M) Ap) 6.1 (M) 1.0 (L) 8.7 (H) 
Cohort! | 16 | 7.8(M) 1.0 8.1 (H) 5.5 (M) 9.1 (H) 
co Clssteem. | ener? ile: | Tei) 9 7.8 (M) 6.4 (M) 9.0 (H) 
Total 31 7.8 (M) § 7.9 (M) 5.5 (M) 9.1 (H) 
Cohort! | 16 | 5.4(M) 1.7 5.7 (M) 2.7 (L) 8.1 (H) 
Sub -Compenesn ies Omernen= | esas. 1s | ele 1.5 5.6 (M) 4.1 (L) 8.2 (H) 
technology resources 
Total 31 5.7 (M) 1.6 5.6 (M) 2.7 (L) 8.2 (H) 
Cohort | 16 3.9 (L) 3.1 5.3 (M) 0 (L) 7.9 (M) 
Sete e mponent le elandhele Cohort2 | 15 | 4.6 (L) 2.6 5.3 (M) 0 (L) 7.6 (M) 
computers 
Total 31 4.2 (L) AES) 5.3 (M) .0 (L) 7.9 (M) 
Cohort | 16 6.5 (M) 1.0 6.4 (M) 47 (L) 8.3 (H) 
SubsComponent: 20) Other Cohort2 | 15 | 6.0(M) 1.4 6.3 (M) 3.9 (L) 8.1 (H) 
technology resources 
Total 31 6.3 (M) 1.2 6.3 (M) 3.9 (L) 8.3 (H) 


ll Sub-component 15 (text sets) is not included in the calculation of Component 5 (Materials) or the overall fidelity scale 


due to missing data for 7 of the 31 schools. 
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Figure 8: Results of Year 3 Implementation Fidelity Scales 
Component 5: Materials 
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Use of materials received the least favorable fidelity score of the five major classroom model 
components. Among the specific, non-technology materials, classtoom libraries received higher 
ratings, but school libraries were rated lower. Ratings of text sets were inconclusive, both 
because of the low response rates from subject-area teachers, and because of the ambiguity of 
whether sutvey respondents distinguished Striving Readers text sets from traditional, school- 
wide text sets. 


Ratings of the fidelity of the use of “high-quality, high-interest materials that are integrated with 
engaging technology and audio resources” reflected the schools’ use of Striving Readers text sets in 
content area classrooms; teachers’ and LITs’ frequency and comfort of use of the school library to teach 
literacy, including extent of collaboration with the school librarian; frequency and purposes for which 
teachers and LITs use the classroom library to teach literacy, and the alignment of library materials with 
students’ interests; the use of other non-technology materials such as vocabulary notebooks, reading 
response notebooks, and other informational texts (other than text sets); and teachers’ and LITs’ 
frequency, focus, and quality of use of technologies (other than the handheld computers), including 
listening centers and media centers. In addition, a full section of the materials fidelity scale was also 
dedicated to LITs’ extent, comfort with, and focus on use of handheld computers for the targeted 
intervention. 


Use of materials received the least favorable fidelity score of the five major classroom model 
components, with an overall score (5.9) at the middle level of fidelity. Among the specific types and 
sources of materials, the use of Striving Readers text sets received the highest rating, indicating a high 
level of implementation, with Cohort 1 schools (with an implementation score of 9.0 at a high level of 
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fidelity) achieving higher fidelity than Cohort 2 (with an average implementation score of 7.9). However, 
these fidelity ratings reflected only one dimension of the use of the text sets (whether principals reported 
that they were being used in content area classrooms); items providing further nuances about how 
content area teachers used the text sets could not be reported on because of low response rates from 
content area teachers.>? In addition, responses about text sets were missing for seven schools. As a 
result, these scores may be misleading at the program level, and for this reason were not included in the 
overall materials scores. 


Classroom libraries also received high ratings, with scores overall and for both cohorts averaging 7.8 
out of 10. At the building level, 8 of 16 Cohort 1 schools and 7 of 15 Cohort 2 schools received ratings 
at the high level of fidelity. School libraries, in contrast, received an overall score (5.7) reflecting medium 
fidelity level, with scores among Cohort 1 schools averaging 5.9, compared with 5.5 among Cohort 2. 
While the libraries at four schools (three in Cohort 1 and one in Cohort 2) functioned at a high level of 
fidelity in relation to the goals of Striving Readers, nine (four in Cohort 1 and five in Cohort 2) were 
rated at a low level of fidelity. One factor that lowered fidelity ratings in this area is that several program 
schools did not have a librarian. This is an important position that can support both classroom teachers 
and LIT by identifying and acquiring literacy materials, collaborating on lesson planning and providing 
resources, and providing direct instruction to students on library skills as well as literacy. Even where a 
librarian was available, however, it is possible that at least for some purposes, teachers and LITs felt less 
need to use their school libraries because their classroom libraries were so well supplied. 


Handheld computers. 


LITs’ use of the Palm Pilots to support the targeted intervention was rated at a low level of 
fidelity. Among the 31 treatment schools, 8 LITs never used them for this purpose, mostly 
because they were used primarily for whole class instruction. 


Fidelity score for use of the Palm Pilots reflected the frequency, comfort and focus with which LITs 
used these devices to provide instruction for the targeted intervention. This sub-component received 
the lowest implementation rating within the materials component, with an overall score (4.2) reflecting a 
low level of fidelity. The majority of LIT’s reported using the Palm Pilots at least occasionally. However, 
the expectation is that they all should be using them to teach literacy to Tier 2 and 3 students during 
targeted intervention instruction; yet, in eight schools—five from Cohort 1 and three from Cohort 2— 
the LITs reported that they were not using them for this purpose at all during 2008-2009. Among those 
who did not, by far the most common reason cited for not using them (mentioned by 7 of the 8 LIT’) 
was that they were being used primarily for whole class instruction. It is notable however that none of 
them reported not having received the units. 


52 This low response rate resulted in part from the fact that self-contained teachers were asked to respond to the ELA 
sutvey, but not the content area survey. 


53 While these devices were often also used during the whole school and intensive interventions, it was the LIT’s use for 
the targeted intervention that was incorporated into fidelity scales, reflecting the primary original purpose of the Palm 
Pilots. 
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Among those LITs who reported using Palm Pilots for targeted instruction on the district-wide 
sutvey, the most commonly used applications were Freewrite, Sketchy, and ik WL, which were used at 
least occasionally by 85% or more of those using the units. All three of these applications were used by 
the large majority of these LITs to support reading comprehension, while iX WL and Sketchy were also 
used to support vocabulary development and Freewrite was used to develop writing skills. 


Palm Pilots were often also used during AMP classes, and by literacy teachers during the 
school day. Both groups used them to support vocabulary development, knowledge of key 
concepts and writing skills; LITs also used them to teach comprehension strategies, while 
classtoom teachers also used them to develop self-directed learning. 


Almost three fifths (58%) of all LITs and other AMP teachers*4 responding to the survey also 
reported using the Palm Pilots during the AMP after-school program. Of these, almost all (92%) used 
them at least monthly, but only one fifth (21%) used them weekly or more. The instructional objectives 
that were most frequently supported with the Palm Pilots during AMP classes were development of 
vocabulary, comprehension strategies, knowledge of key concepts, and writing skills, each of which was 
supported with activities on the units by two thirds or more of the instructors who reported using them. 


A very similar proportion of surveyed literacy teachers (57%) also reported using handheld 
computers to teach literacy during the 2008-2009 school year. Among those who were using them, the 
largest proportion (41%) felt moderately comfortable (rated as a 3 on a scale ranging from 1 = not at all 
comfortable to 5 = very comfortable) doing so, while almost one third felt comfortable using the units 
(31% rated their comfort as a 4 or 5). Even amongst those who were using them, however, over a 
quarter (27%) still felt uncomfortable with them (rated as a 1 or 2). The majority of these respondents 
(70%) reported that they used the Palm Pilots to support differentiated instruction, while the most 
commonly supported instructional objectives included vocabulary development and demonstrating 
knowledge of key concepts, each of which were cited by three fourths (76% and 74%, respectively) of 
those using the devices. In addition, three fifths of these teachers also reported using the units to 
support writing skills and to develop students’ self-directed learning (61% and 59%, respectively). This 
latter objective is of particular interest given its alignment with teachers’ efforts to provide differentiated 
instruction, and with the program’s goal to promote a transfer of responsibility from the teacher to the 
student (“gradual release”). 


Many case study instructors believed that students found the Palm Pilots engaging. At least 
some had begun to take advantage of their added value for promoting instructional goals, but at 
other times they were used simply as word processors. 


Within the case study schools, among teachers and LITs who were using the Palm Pilots, there was a 
general consensus that students find them engaging. But beyond the “wow” factor of new technology, 
at least some teachers have realized that using the devices can amplify the benefits of certain 
instructional activities. As one teacher commented during a case study focus group, for example, “T 
think the Palms ...[have] helped us because they really narrow down, it’s not pencil and paper anymore, 


54 In some of the larger schools where the LIT is unable to teach all of the AMP classes, classroom literacy teachers 
teach the class for grades 7 and 8. 


84 


it’s them thinking and you really have to limit yourself on the Palm, so they have to get to the nitty gritty. 
They can’t write like you would on a sheet of paper and I think that makes the difference.” In another 
example, a teacher described using the Palms to complete graphic organizers for a KWL activity: “The 
students were used to doing it in the form of a written graphic organizer, but once they got their hands 
on the Palm Pilots, and saw that they can do it, and half of their short phrases were already programmed 
in for them, they really got a kick out of that. I actually see more input and more ideas from the 
students.” In other cases, teachers have used the devices for perhaps more prosaic purposes—such as 
drafting and printing class papers—in which they are serving the same function as a desktop computer. 
Nevertheless, since part of the purpose of acquiring the Palm Pilots was to have a less expensive means 
of making computers accessible to more students, these kinds of applications are still appropriate. 


Activities during observed lessons at the case study schools at which students used the handheld 
computers included a class working to make connections to a novel they were reading on iKWL; 
identifying character traits on PicoMap; and, following a whole-class activity where the LIT modeled the 
process, breaking up into small groups to use Sketchy to represent story development and practice 
visualization skills by drawing characters and scenes. Among numerous other observed applications of 
the Palms were students, working individually, learning to use Inspiration to create graphic organizers 
representing cause and effect; recording notes about problems and solutions; and creating Venn 
diagrams to compare and contrast vocabulary words. 


Challenges to using the handheld computers included the same factors that hindered 
differentiation in general. Some teachers also struggled with ways to differentiate instruction on 
the devices when the whole class was working on them together. 


Among the challenges that case study classes encountered in these activities were some that related 
to the instructors’ continuing learning process on use of differentiated instruction, and others that were 
more specific to the devices themselves. For example, in one activity, the LIT oversaw one group as 
they worked on the Palms, but the LIT provided limited feedback to students. It may have been that the 
LIT saw that the students were comfortable with the process (including the technology), but did not 
focus enough on providing scaffolding around the content of the students’ work. It is also possible, 
however, that the activity was not conceptually challenging enough for this group. In other cases, there 
was no apparent structure to the lesson to facilitate differentiation, for example because all groups were 
doing the same activity and working with the same materials. 


Comments by some teachers during case study focus groups revealed that more fundamental 
misunderstandings about how to use the Palms existed. Because students enjoy working with them so 
much—and possibly, in part, because teachers’ were still not fully comfortable with using them for 
differentiation—most classrooms have decided to involve all students when using the handheld 
computers, rather than providing them as instructional supports to particular students. Some teachers, 
however, perceived this situation as creating an obstacle to differentiation. In another observed lesson, 
an activity intended for small, leveled groups was restructured as a whole-class activity because students 
had difficulty understanding how to use the software. In this example, however, the problem was clearly 
circumstantial and likely to be resolved once students became more comfortable with the technology. 
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Although Palm Pilots were distributed to all schools, classroom teachers who were not yet 
using them reported that they either had not yet received them, or that they still did not feel 
comfortable using them. A quarter of respondents felt that they were not “worth the trouble.” 


Among those classroom literacy teachers responding to the district-wide survey, 30% of those who 
did not use the handheld computers reported that they had not yet received them. It should also be 
noted that many teachers were new to the program and never received the initial training. Given that the 
program had distributed the devices to all schools, it is possible that some of this had to do with internal 
distribution issues within particular schools. The most common reason for not using them, however 
(cited by 37% of those not using the units) was that they had “not yet received sufficient professional 
development to feel comfortable using them.” (Although 80% of the respondents citing this reason 
reported that they did receive training, it is not clear how much of the offered training these respondents 
attended.) The Striving Readers technology specialists confirmed that teachers are often still uncertain 
about how the devices should be integrated with instruction, and adapting to this new instructional 
modality adds further to their time constraints. About a quarter (26%) of surveyed classroom literacy 
teachers felt that the units “do not offer sufficient added benefit compared to traditional media (e.g. 
print, paper, and pencil) to be worth the trouble.” Indeed, the project director has acknowledged that 
the use of technology—and of the Palm Pilots in particular—to support differentiated instruction has 
been one of the greatest challenges facing Striving Readers staff. As exemplified above, this is true both 
because differentiation itself has required time to become established, and because use of the 
technologies presents challenges of its own. School coordinators also noted that throughout Year 3, 
teachers and students were still becoming acclimated to their use. As a result, according to one 
coordinator, some lessons focused more on how to use the technology than on how to use it to 
strengthen literacy-based instructional goals. 


Where Palm Pilots were being used, classes were often still focusing on learning how to use 
the technology itself. Because LITs were a critical link in informing technology coordinators of 
problems and in supporting the development of lesson plans, schools where the LIT did not 
advocate strongly for technology, or had mote limited computer experience, had greater 
difficulty integrating the Palm Pilots. Some LITs, in turn, were dissatisfied with technology 
support services, which may have been hampered by ambiguity about delineation between 
district- and school-based technology coordinators’ roles. Frustrations with these challenges 
might have reduced teachers’ motivation to use the devices. 


Clearly, the introduction of an unfamiliar instructional modality would be expected to require 
considerable training, support, and experience before optimal implementation is achieved. In 
recognition of this fact, the Chicago Striving Readers program has provided substantial hours of 
professional development in the use of handheld computers, beginning in program Year 2. However, 
with the exception of a few pilot schools at which they were introduced in the spring of 2008, the Palm 
Pilots had been introduced to most schools relatively recently at the time that the Year 3 surveys and 
interviews were conducted, and even at the case study schools, teachers consistently indicated that they 
were just learning how to use them. Many teachers reported that they were still at the stage of getting 
the entire class acquainted and comfortable with the Palms. Moving past this introductory stage would 
likely be necessary before instructors could use the devices for more complex applications that might 
better support differentiation. 
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Although it might reflect the fact that they were still relatively inexperienced with the Palm Pilots, 
during case study focus groups, many teachers said they felt that professional development had been 
insufficient, both in terms of how to operate the devices and how to use them to support literacy 
instruction. A number of teachers also reported that the training they received had used a different 
model of handheld computer than the ones that were distributed—a surprising finding, given the project 
director’s assurance that this was not the case. 


In order to ensure adequate support to teachers as they learned to use the Palm Pilots, the school 
coordinators and the district technology coordinator all described the LITs’ role as liaisons between 
them and classroom teachers as being critical. In addition to making sure that the technology specialists 
were aware of any technical problems teachers may have encountered, the LIT’ also were directly 
responsible for supporting the development of lesson plans that used the devices. One LIT who agreed 
with this point put it succinctly: “If they have the support of the LIT then I think they are more apt to 
use the Palms.” 


Precisely because of the importance of the LITs’ roles in this regard, the technology specialists felt 
that a few LIT’s who were not advocating for technology at their schools as much as they could, and/or 
who still needed basic computer instruction themselves, were the “weakest link” in adopting technology 
at their schools. And because time constraints prevent LITs from serving the upper grades in larger 
schools, some staff have noted that use of the devices occurs less frequently at grades 7 and 8. 


While the technology specialists expressed concerns about the LITs’ role, in the case study, several 
LITs expressed dissatisfaction with the adequacy of support services for resolving technological issues, 
while teachers reported that the Palm Pilots were frequently not working. Project leadership has 
continued to work to resolve such impediments, in part by strengthening and clarifying the roles of the 
technology specialists. (An area about which these specialists still express concern is ambiguity about the 
delineation of their responsibilities with those of the school-based technology coordinators—a situation 
that will likely become crucial when the grant expires.) 


Unfortunately, the various challenges to using the handheld computers has generated considerable 
frustration, and may have contributed to reduced conviction about their value on the part of a number 
of teachers. For some teachers, the value added by the Palms above and beyond that of desktop 
computers—or even paper and pencil—was still unclear. The comments from classroom teachers 
during two of the case study focus groups state this rather pointedly: 


I just think between a keyboard not working and the Palm not turning on, and the responsibility of keeping track 
of them and charging them and putting them away and how do you to put it on the computer so that they can see 
what you're doing, all of that takes three times as much as it does for them to actually do something on the Palm. 
And what I’ve seen so far is that they’re making either some kind of a PowerPoint on ViewPoint or a KWL and 
I just think if there’s money out there, the technology is great, but all I’ve really heard is, ‘it’s engaging.’ 


[T]he Palm Pilot computing power is a lot less than the desktops that we already have. ... [I]n our case, we 
already have ready access to computers and desktops. I could see these Palm Pilots with their keyboards being a 
lot more useful or appropriate in a setting that did not have access to desktop computers, Internet, or things like 
that. The fact is, we don’t have access to the Internet on the Palm Pilots, anyway. 
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Other technology resources. In addition to the handheld computers, the Striving Readers 
model incorporates several other types of technology that are provided through the program. These 
include classroom media centers (three computers and a classroom printer); as well as classroom listening 
centers, where students can listen to audiobooks and models of fluency (many listening centers used 
Playaways for this purpose). The AMP program utilized the computer-based AMP software program, and 


the classes also often had access to many of the other technology resources mentioned above. 


Both media and listening centers were used by most literacy teachers. However, media 
centers were used even more frequently and were better integrated into the curriculum, in part 
because of a shortage of materials for the listening centers, and in part because of greater 
familiarity with the more traditional technology. 


Although both were used by the large majority of classroom literacy teachers, media centers, which 
utilize more traditional technologies with which teachers are more familiar, were used somewhat more 
frequently than the listening centers. Part of this difference was apparently related to access: Among the 
15% of sutveyed teachers who were not using the listening centers, about half—8% of all respondents— 
said they did not have one. A somewhat smaller proportion of teachers (5%) said they did not have a 
media center, and a total of only 6% said they were not using one. In addition, the district technology 
coordinator attributed the less frequent use of literacy centers to a shortage of materials (such as 
audiobooks) with which to use them. Nevertheless, the program director has pointed out that, if 
teachers are resourceful, the centers can still be used productively without the need for materials 
provided by the program. For example, many classrooms and/or school libraries already have their own 
books on tape or CD, or they could borrow them from public libraries. 


Program-wide, media centers were used quite regularly, with almost 4 out of 5 teachers (78%) 
reporting that they used them at least once a week. Among LITs, 90% reported using them for the 
specific purpose of targeted instruction for Tier 2 and 3 students, including 60% who did so at least 
weekly. In addition, more than three fifths of all Striving Readers principals (63%) believed that the 
media centers were “thoroughly integrated” into their school’s literacy curriculum. 


Despite some concerns about access, the listening centers were nevertheless generally well used: the 
large majority of teachers (85%) reported using them, including almost half (48%) on at least a weekly 
basis. (Among those LITs responding to the question, 80% were using them for targeted instruction, 
including one in four who were doing so at least weekly. However, the LITs who did not respond to 
this question may have skipped it because they were not using the centers. If so, this would imply that 
only 53% of LITs were using them.) Only two fifths (41%) of interviewed principals felt that the 
listening centers were “thoroughly integrated” into their school’s literacy curriculum, but the remainder 
felt that they were at least “somewhat integrated.” 


During the case study, class observations presented numerous examples of teachers integrating 
technology into their literacy instruction. These included a class in which students developed their 
listening and comprehension skills by using their Playaways to read along with novels that were leveled 
to their ability but self-selected according to their interests; students then responded to discussion and 
reflection questions that each had chosen from the Readers’ Notebook. In another literacy lesson, 
students worked in small groups studying grammar on the Internet. Technology use was also observed 
in content area lessons, including, for example, in one science class in which students worked in pairs to 
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research cell structure on the Internet, and in a seventh-grade class in which students prepared for a 
debate by researching their topics on the Internet and using the computers to prepare their evidence 
cards. 


Technology is also expected to be a significant component of the AMP program, and this too was 
supported by the case study observations, which often included efforts to achieve differentiation through 
concurrent small-group activities, many of which depended on technology. Examples included partner 
reading using online materials, followed by the partners asking each other questions about what they had 
read; students conducting Internet searches; and students conducting read-alongs while listening to their 
Playaways. During these observations (as with the class observations during the school day), while it was 
generally not apparent to the observer whether the various activities were appropriately assigned 
according to each group’s strengths, needs, and interests, differentiation through technology use 
appeared to be further supported by virtue of the LIT’s individualized involvement and interaction with 
the students. 


Classroom libraries. 


Classroom libraries were used regularly and almost universally to support literacy instruction, 
and instructional staff were enthusiastic about their value for supporting all aspects of the 
ptogram. Many teachers commented that they helped to foster a love for reading. 


According to the spring surveys, virtually all teachers (98%) and almost all LIT's (93%) had used the 
classroom libraries to support their literacy instruction. Among LITs, over one third (37%) had done so 
on a daily basis, and the large majority (80%) at least weekly. Classroom literacy teachers used them even 
more regularly: over half (56%) had used their classroom library on a daily basis, and an even larger 
majority than among LITs (86%) did so at least weekly. (It is notable that both teachers and LITs used 
the school library considerably less frequently than this, which may be related to the previously stated 
speculation that the quality of classroom libraries may have reduced the perceived need to use school 
libraries. Specifically, among literacy teachers and LITs, respectively, only 51% and 31% used their 
school libraries at least weekly.) 


During case study interviews and focus groups, teachers and LITs commented on the rich quantity 
and variety of books available through the libraries—a perception that was strongly confirmed by 
district-wide surveys: 80 percent of teachers felt that it was “very true” that their libraries included a 
variety of materials appropriate for different reading abilities, and 77 percent said it was “very true” that 
the materials appealed to a range of student interests. Indeed, the classroom libraries made possible 
through the Striving Readers program were extolled by teachers and LITs alike for their value in 
supporting multiple aspects of the program model. The variety of subjects, genres, and levels of 
materials supported small-group and whole-class leveled reading activities. Case study teachers also 
noted the benefit of being given funds instead of particular books, as it allowed them to select according 
to their classroom needs and the interests of their students; indeed, 80% of surveyed literacy teachers 
reported that they used interest inventories to guide their purchases for their libraries. This attention to 
high-interest materials helped build student excitement and motivation to read and encouraged 
independent reading. 
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Teachers’ enthusiasm for the value of the classroom libraries was captured by one teacher’s 
comments during a case study interview: “Having been here for three years, ’ve seen change over the 
years, considering the first class I had and their interest in independent reading. ... It was better last 
year, and the kids I have this year are even more into it; so I can kind of see them going through Striving 
Readers and having access to all of those books and they want to read more, and they do. Last year, that 
wouldn’t have happened.” 


Some teachers found it difficult to identify materials for classroom libraries that adequately 
challenged their higher-achieving students, and delays at the school level in filling book orders 
also made it more difficult to use them to support student assessment. Despite the variety of 
these reading materials and their appeal to most students, efforts to foster a love of reading were 
sometimes hindered by families and cultures that do not encourage it. 


Even with this level of enthusiasm, the classroom libraries were not without their challenges. One 
problem, which in one sense may be the kind of problem that most teachers would like to have, was that 
a number of case study teachers indicated that they did not have enough shelf space for all the materials. 
More significantly, at least one case study teacher described having some difficulty providing appropriate 
reading materials for the higher achieving students—a concern that paralleled those voiced by quite a 
few case study respondents who felt that Striving Readers does not adequately challenge their high- 
achieving students. Given that Striving Readers teachers are responsible for acquiring their own 
classroom library materials (with support from program funds), and project leadership’s contention that 
the difficulty using standardized program materials for more advanced students reflect a limited 
repertoire in how to use them rather than a limitation of the materials themselves, these concerns seem 
to indicate a need for further training. 


Another difficulty that case study teachers said they encountered was the time it took to order books 
through the program. This situation exacerbated the problem (discussed above) with being able to 
implement student assessments in a timely fashion so that they could be used to inform instruction. 
Delays in book orders (which are the responsibility of each school) made it more difficult to tailor library 
purchases to such assessments. Some teachers noted that they had had to purchase books prior to the 
start of the year so that they would arrive early enough during the academic year. 


Content area teachers also received funds from Striving Readers to support acquisition of reading 
materials. Apparently misinformed, some of the content teachers in the case study indicated that they 
believed that the program requires them to use a portion of their funds to purchase materials that are 
more literacy specific than content specific, such as fiction books, graphic organizers and books on tape 
(for the listening centers), as well as genres like fantasy and science fiction that appeal to students’ 
interests but do not support the content as well. In fact, the leadership team reports that it encourages 
content atea teachers to purchase non-fiction materials primarily. Encountering challenges that would 
be more generally applicable to both content area and ELA teachers, at a focus group at one case study 
school, self-contained teachers noted that they had had difficulty finding materials for their higher level 
readers. They also encountered trouble with long delays between ordering and receiving the texts. °° 


55 The latter comments were made in reference to acquiring text sets. However, this term is commonly used to refer to 
sets of materials developed by teachers, that are not the same as the sets that were specifically developed and 
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Finally, the use of classroom library materials to motivate students and foster a love for reading was 
hampered by a problem that was more broadly related to the larger goals of the Striving Readers 
program. The senior literacy advisor observed that some cultures and families do not encourage reading, 
and that there was a lack of initiative among teachers and LITs to explore strategies to overcome this 
orientation. For a program designed to develop literacy skills, these are clearly issues of real concern. 


Text sets. 


Text sets were used to support content area instruction in almost all schools, and were also used 
in almost all literacy classrooms, although in many cases they were not used on a regular basis. 
Some teachers praised the quality of the materials but felt there was not enough, while others 
did not feel that they adequately supported the content areas. 


In addition to the use of the text sets in content area classrooms, as reported by principals, these 
materials were used quite regularly in literacy instruction. (This aspect of their use was not included in 
the fidelity scales because program leadership indicated that it was not an explicit expectation of the 
model.) Almost all LITs (93%) responding to the survey reported using text sets to teach literacy. While 
the large majority (80%) reported doing so at least monthly, only about a quarter (27%) did so at least 
weekly. A similar proportion of classroom literacy teachers (90%) reported using text sets to teach 
literacy, with over half (54%) doing so at least weekly. In self-contained classrooms, integration between 
literacy and content area instruction often occurred fairly seamlessly, as the teacher (and LIT) were more 
prepared to use informational texts as the basis of literacy activities, such as read alouds and independent 
reading. In content area lessons, principals pointed out, text sets are often used to support project-based 
learning. Some teachers would divide the text set materials for a particular topic among small groups for 
a jigsaw activity. Presumably this distribution would be based on reading level, although this was not 
explicitly stated, and may have been more difficult for content area teachers (who may have less access to 
reading assessments or to the LIT) than for self-contained teachers to do. 


Challenges hindering use of the text sets included those that were specific to content area teachers 
and classes, and others that were more generally relevant. In some case study interviews, content area 
teachers indicated that the text set materials were high quality but that there were not enough of them, or 
that that they were distributed at their school in only one subject, or that they did not receive the text 
sets at all. Some content area teachers felt that the text sets touch on content only superficially and focus 


distributed by the Chicago Striving Readers program. While interview and survey questions related to the Striving 
Readers text sets provided clarifying definitions, it is apparent that some respondents nevertheless remained confused 
about this distinction, as was the case with these comments. 
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more on literacy, while others felt that the materials in the sets do not align well with the curriculum in 
their subject. 


Even the project director conceded that, as of the third program year, the text sets had not proven as 
useful as they could have, largely because they were not as far along in implementation as they should 
have been due to delays in distribution during Years 1 and 2. Professional development was provided 
around the use of text sets and classrooms began using them for PRC2. Further development of 
additional sets that took place during summer 2009, and additional training and support during the 
current school year, were expected to improve implementation of this component. 


Content Area Instruction. 


Striving Readers schools seemed generally very supportive of integrating literacy into the 
content areas. There was recognition that the disciplines are mutually supportive, although this 
may be less true in content area lessons based on inquiry-driven, experiential learning. 
Differentiated instruction also appeared to present an even greater challenge in content area 
classes. School coordinators felt that content teachers who are mote open to less traditional 
techniques tended to be more successful at integrating literacy. 


On principle at least, there appeared to be fairly broad support for the concept of integrating literacy 
into content area instruction. During program-wide interviews, principals explicitly expressed support 
for the program’s interdisciplinary approach and practices that make it possible for content area teachers 
to support literacy. One case study principal was gratified that “all teachers know that no matter what 
subject they teach or department they belong to, literacy instruction is everybody’s job.” Content area 
teachers at the case study schools also seemed broadly accepting of the goal of integrating literacy. More 
important, many of their comments demonstrated that they were specifically aware of the value of such 
efforts. While staff often alluded to the value of integration to support literacy, some teachers also talked 
explicitly about the corollary that developing good reading skills is necessary to acquire content. During 
a case study focus group, one content teacher described some specific examples of this benefit: “[As a 
result of the program,] I think that my kids are better at picking out the main idea and summarizing— 
especially in science and social studies ... just from focusing on looking at the text structure, the 
headings, and that kind of thing.” Case study teachers pointed out that because the reading students do 
for Striving Readers includes a lot of informational text, and because of the focus on vocabulary through 
Marzano’s vocabulary and Words Their Way word study processes, they are developing their grasp of 
content area vocabulary and acquiring more content knowledge. 


Some of the strongest examples of support for integrated instruction were seen at two K—8 case 
study schools, where principals described having expanded the Striving Readers program school-wide, 
across all subjects and grades, creating a high level of coherence in their curriculum. In at least one of 
these schools, the school-wide motivation to enlist in this initiative was reinforced by improving test 
scores. This motivation was likely a significant factor in its success. 


The recognition of the value of integrating literacy and content, however, was not necessarily a 


predictor of success in its execution. In case study interviews, some content area teachers found it 
difficult to incorporate literacy into content lessons when these lessons utilize inquiry-driven, experiential 
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learning—which is also often included as an instructional goal. One teacher summarized the situation by 
stating that “by design, both math and science are exploratory works—which leads less to a literacy 
approach than to a hands-on physical approach.” 


During case study observations of subject area lessons, some content area teachers seemed to 
struggle—even more so than ELA and self-contained teachers—with techniques for differentiating 
instruction. After providing instructions for a particular activity, although students often seemed familiar 
with the tasks, the teachers’ instructions in several instances were not followed through to make certain 
that students understood the assignment. And in some observed content area lessons, the entire period 
seemed devoted to whole-class instruction. While other groupings may have been used in those classes 
on other days, it would seem unusual for an appropriately designed Striving Readers lesson to include a 
whole period with no small-group activities. In some whole-class activities observed during subject area 
lessons, all students were seen to be working individually on the same task with the same instructional 
materials. Activities with similar, class-wide consistency were also observed during language arts lessons 
(and other content area lessons) when they were structured in small groups, but in these cases the small 
group structure afforded a better opportunity for instructors to provide intervention and guided 
instruction to specific students. 


During fall 2009 interviews, the school coordinators acknowledged that Striving Readers had been 
expanded to the content areas only to a limited extent during the 2008-2009 school year. In contrast to 
expansion of the program to the upper grades, where success depended strongly on the availability of the 
LIT to those grades and, therefore, on school size, coordinators suggested that the success of integration 
into the content areas depends more on the individual teacher. They felt that teachers who are more 
comfortable with new techniques and less committed to traditional instructional strategies tended to be 
more successful at integrating literacy in the content areas. While it is yet to be seen, it is also possible 
that, with the district’s shift to a departmentalized structure as of the current (2009-2010) school year, 
teachers with prior experience in self-contained classrooms who ate now teaching content area subjects 
may be well positioned to provide instruction that integrates literacy. 


Component 6: Professional Development 


Fidelity scale results for professional development are presented in Table 20 and Figure 9. 
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Table 20: Results of Year 3 Professional Development Model Implementation 


Fidelity Scales 


Fidelity Component Cohort | N Mean Score Standard Median Minimum MED danelan 
Deviation Score Score Neto] a= 
Cohort | 16 5.5 (M) 8 5.6 (M) 4.1 (L) 6.8 (M) 
Component 6: 
; Cohort 2 15 6.5 (M) 9 6.7 (M) 5.1 (M) 7.8 (M) 
Professional Development 
Total 31 6.0 (M) 1.0 5.8 (M) 4.1 (L) 7.8 (M) 
Cohort | 16 4.8 (L) J 4.9 (L) 3.0 (L) 5.8 (M) 
Sub-Component 21: Whole- 
shel eb Cohort 2 15 4.9 (L) 6 5.0 (L) 3.7 (L) 5.7 (M) 
Total 31 4.9 (L) all 4.9 (L) 3.0 (L) 5.8 (M) 
Sub-Component 22: PD for Cohort | 16 6.3 (M) 1.3 6.3 (M) 3.3 (L) 8.8 (H) 
targeted and intensive Cohort 2 15 8.1 (H) 1.7 8.4 (H) 5.0 (M) 10.0 (H) 
a cc Total 31 | 7.2(M) 18 7.5 (M) 3.3 (L) 10.0 (H) 


Figure 9: Results of Year 3 Implementation Fidelity Scales 
Component 6: Professional Development 
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Program implementation for the professional development model was based on two fidelity scales: 


one reflecting training activities in support of the whole-school (blended) model, and one reflecting 


activities that were specific to the targeted and/or intensive interventions. It should be noted, however, 


that in the Chicago Striving Readers program the large majority of professional development that is 
designed to develop the targeted and intensive interventions is built into the training activities that 
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support the blended model more broadly. For this reason, fidelity scores from activities specific to these 
interventions should not be considered in isolation; rather, fidelity ratings for all activities combined 
(including those specific to the targeted and intensive interventions) should be interpreted as representing 
fidelity for the targeted and intensive intervention training. 


For these scales, scores reflected the average attendance rates across all of the primary-targeted staff 
for a particular category of sessions, across all staff who were expected to attend. While many of these 
sessions could be attended by other staff as well (for example, principals could attend trainings targeted 
primarily for teachers), this additional attendance was not incorporated into the fidelity ratings. In 
addition, there were several types of professional development that were not incorporated into the 
fidelity scales because they were not considered as core training by program leadership. Categories of 
professional development that were reflected in the scales included: 


¢ Monthly professional development (principals) 
e Weekly meetings with coordinators (LITs) 

e Annual summer institute (teachers) 

¢ Monthly Saturday seminars (teachers) 


¢ Quarterly follow-up institutes (teachers) 


Professional development fidelity scales also reflected the extent to which the primary participants 
considered each session useful, the amount of impact they felt it had on their comfort with each teaching 
practice, or both. Detailed definitions of the professional development fidelity scales are provided in 
Appendix E; while the complete professional development schedule for the 2008-2009 school year is 
shown in Appendix G. 


Teachers and school administrators expressed appreciation for the inclusiveness and timing 
of the Striving Readers professional development program. Principals were a great help in 
bringing LITs on board with program philosophies, and as a result of LITs’ role to provide 
feedback about teachers’ needs, many teachers have also found the training to be highly 
responsive. 


Fidelity of the training was limited by two primary factors. Although most LITs were highly 
involved in weekly meetings with coordinators, attendance by teachers has been problematic, 
and often not sufficient to prepare them to fully implement the model. At the same time, some 
staff feel that certain sessions have been repetitive. Leadership is still exploring ways to further 
strengthen its efforts to provide differentiated training. 


Results of these scales show that implementation of the professional development for the whole- 
school intervention—including attendance and perceived impact—occurred at a low fidelity level, overall 
and for both cohorts. At the building level, there were fewer schools at low fidelity among Cohort 2 
than Cohort 1 (9 out of 15 compared with 12 out of 16). Training for the targeted and intensive 
intervention models was rated at a medium level of fidelity overall and for all schools except four Cohort 
1 schools, which were rated a low fidelity. However, these higher ratings for targeted and intensive 
intervention training (in which the whole-school ratings are embedded) primarily reflect stronger 
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attendance by LITs at their weekly meetings with coordinators. Even in the latter area, however, while 
Cohort 2 schools received ratings at a high level of fidelity (overall and for 9 out of 15 schools), only one 
Cohort 1 school was rated at high fidelity, and three were rated at low fidelity. 


Principals, most of whom have had extensive professional development from other sources, have 
been generally positive about the training offered through Striving Readers, and have expressed 
appreciation for being included in these sessions. During case study interviews, teachers also expressed 
appreciation for the program’s responsiveness to specific requests for learning opportunities. Teachers 
commented that the summer sessions were helpful because they helped prepare them before the school 
year started. Case study teachers also said that they appreciated the summer sessions, because they 
enabled them to determine what kind of training they needed and choose which sessions to attend. 
These comments seem to indicate a misunderstanding about attendance requirements, however, as the 
project director had stated that all participants were expected and strongly encouraged to attend all 
sessions. This misunderstanding may have been one reason for low attendance rates, at least during the 
summer session. In addition, the Summer Institute convened shortly after major changes in program 
leadership occurred, including appointment of a new project director and school coordinators, and the 
new director felt that this situation generated some initial skepticism among school staff until the new 
leadership was able to re-establish trust. Project leadership believe that attendance rates at Saturday 
seminars had also been adversely affected by teachers’ personal obligations and need for free time. 
(During Year 4, the program has moved these sessions to coincide with district staff development days 
in an effort to improve attendance.) 


As liaisons between teachers and the district team, LITs served an important role in the professional 
development program by providing feedback to inform the types of training that were needed, as well as 
identifying which teachers needed particular assistance. Many LIT, who often entered this position 
directly from the classroom, had had extensive experience as classroom teachers. While this experience 
ensured strong qualifications for the role, it often had the additional effect, according to the senior 
literacy advisor, of crystallizing their approach to instruction. As a result, program leadership indicated 
that many LITs initially were not eager to adopt the methods and philosophies of the Striving Readers 
program; however, the principals have been a major support in helping bring the LITs on board. One of 
the primary foci of the professional development program during the third program year was to set 
expectations and create a shared vision. These efforts helped to facilitate communications among 
teachers, LIT's, and the project team, thereby strengthening the structured training that district 
coordinators provided to the schools. 


In addition, the project team also made efforts to differentiate the professional development 
opportunities according to the needs of different staff members. Consistent with the program 
philosophy of data-driven instruction, project leadership uses its own implementation rubrics to monitor 
program implementation at the schools. These rubrics are used primarily to enable the district team to 
provide better targeted support and professional development. Differentiation of training included 
providing sessions designed specifically for new teachers—a critical consideration given high turnover 
rates in the district—as well as specific sessions addressing content area instruction. Because their 
numbers are smaller, training for LITs was easier to differentiate, and these sessions were conducted to 
address LITs’ individual needs and comfort levels. In addition, during the third program year, the senior 
literacy advisor met individually with principals from approximately 8 to 10 of the Striving Readers 
schools in an effort to help link the program to other aspects of their curricula. These meetings 
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originated by principal invitation, or as a result of conversations with the principals during project 
meetings. 


In order to further individualize the training, LITs and classroom literacy teachers collaborated on 
conducting action study groups during the third program year. Each group would focus on one of the 
key program components, selected according to the needs of students and teachers. The group would 
fully implement the selected component in their classrooms, and follow up by examining the successes 
and challenges encountered and exploring its impacts on students. Findings from each study group were 
then presented during training sessions. 


Unfortunately, however, most of the project team’s focus on providing differentiated professional 
development, according to the project director, did not start in earnest until late in the third project year 
(and she acknowledges that the program still needs to better differentiate the principal trainings). 
Indeed, at the case study schools, both principals and teachers have commented frequently that they 
thought that certain professional development activities were repetitive of past sessions. One principal 
even went so far as to say that the teachers felt “insulted” as a result. It seems likely that this impression 
might be a significant factor in the low fidelity ratings for professional development, both in terms of 
attendance and of participants’ impact ratings. The perception of redundancy in the training program 
might also explain why fidelity scores were lower among Cohort 1 schools, where staff have been 
attending trainings for an additional year. It should also be noted that several Cohort 1 schools are high 
performing Autonomously Managed Performance schools. Because of their students’ high performance, 
teachers in these schools sometimes feel less compelled to attend additional training. In addition, 
attendance from some Cohort 1 schools is further impacted by practical considerations: since these 
schools are larger, on average, they often send only one teacher per grade level to professional 
development, and this teacher would then turnkey the training to her colleagues. While this model has 
helped to further disseminate program training, it results in lower official attendance rates. 


One final consideration that appears to have been a soutce of frustration for at least some 
participants—although it is not clear whether it is a problem generally—has to do with the extent to 
which staff are coming away from the extensive program of professional development with the “big 
picture.” The teachers from at least one case study school expressed frustration that they never received 
a summary of what the Striving Readers model is and what is expected of them. These teachers 
characterized their training as providing them with “all little bits and pieces that we put together,” and 
they still do not feel sure they are meeting expectations. Even veteran teachers feel that the sheer 
number of techniques and the recency with which they were introduced has made it challenging, not 
only to implement them correctly but to feel confident in knowing when and if they should be 
incorporated at all. As one principal explained, “Often times teachers might feel a little overwhelmed 
knowing when to use the different strategies, or often feel trapped that they have to cover all of those 
different techniques.” Instead, they may opt to stick with those techniques with which they feel most 
comfortable. 


Broader Considerations of Program Implementation and Management 


Maintaining the motivation and ownership of all stakeholders has been an ongoing challenge. 
Motivation is increased when test scores improve, but the ISAT may be less sensitive than some 
of the program-based assessments. The time it takes for change to become established, 
combined with difficulty detecting impacts, can reduce ownership at all levels. 
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During Year 3, the project director worked to strengthen collaboration within the project 
team by building trust and establishing clearer roles and responsibilities, while school 
coordinators’ involvement in dissemination efforts helped them to find a voice. These efforts 
helped to make the district team more professional and focused. 


While there are myriad practical challenges to implementing a program as broad in scope as 
Chicago’s Striving Readers, a theme underlying all of these challenges, that has been cited consistently by 
all members of the program leadership team, has been the challenge of maintaining the motivation and 
ownership of all stakeholders. In some cases, this effort has been greatly facilitated by the 
encouragement that staff received from improving test scores. On the other hand, in cases where staff 
have not yet seen enough evidence of improving scores, they may be easily discouraged. No teacher is 
willing to think of the students in her current class as test cases to be used to improve instruction for 
future cohorts. There is an inherent dilemma in this, however: an educator’s personal concern for each 
individual student would never be questioned, but the focus on the personal may be difficult to reconcile 
with a focus on the big picture. Some teachers have been encouraged by evidence of progress from 
more finely-grained assessment instruments even when it has not been evident in standardized test 
scores. Even where other assessments may show evidence of impact, however, the reliance on the ISAT 
for the evaluation—the only available outcome measure that is universally implemented in Chicago at 
these grades—may discourage staff at all levels from putting as much stock in other assessments. 


The time it takes for systemic change to become established, combined with difficulty in detecting 
impacts, have likely contributed substantially to frustrations and resulting interpersonal tensions. These 
tensions have been manifest not only as skepticism or impatience among teachers, but also with some 
school administrations that were reluctant to take ownership of the program. In addition, major changes 
in program management during Year 3—including appointment of a new program director in April 2008 
and turnover among school coordinators—have further challenged the cohesiveness of the project team. 
In response, the new program director has worked closely with the all members of the team to agree on 
a clearer set of roles and responsibilities in an effort to strengthen collaboration. In addition, school 
coordinators’ involvement in recent program dissemination efforts through presentations at conferences 
of the International Reading Association has helped them to “find a voice,” while a well-established 
relationship between the program director and senior literacy advisor has supported a strongly trustful 
collaboration. As a result of these developments, according to the literacy advisor, the district team 
became more professional and focused during the third program year. 


Implications for Impact Analyses 


As previously discussed, assuming that a program model is in fact effective (and that its 
essential components are accurately identified)—it would be expected that program impact 
should vary with the fidelity of implementation. Implementation of several program 
components was rated at high fidelity, but it was significantly limited for many other 
components. Obstacles to implementation included lack of ownership from key stakeholders, 
low attendance at training sessions, and high staff turnover rates at many schools. 


Even for components with high fidelity, ratings may be inflated due to respondents’ 


imperfect understanding of the model, and variations in implementation among schools and 
sub-components would still attenuate the strength of program impacts. Variability in program 
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components that are part of the blended model would tend to reduce impact for all participants, 
while low participation and attendance in the AMP program would tend to reduce impact 
specifically for Tier 3 students. 


As has been discussed in relation to previous program years, assuming an effective program model, 
program impact would be expected to vary with the intensity and fidelity with which the model was 
implemented. Several aspects of Chicago’s Striving Readers program were rated, according to teacher, 
LIT and principal reports, as having been implemented with high fidelity during the third program year. 
Notable among these were the overall targeted and intensive intervention models. However, even 
among the highest-rated program components, there was substantial variation among schools and 
among sub-components in many areas, and such variations would continue to attenuate the strength of 
any impacts that could be detected for the program overall. In addition, to the extent that the fidelity 
measures rely on participants’ understanding of the model to be able to provide accurate self- 
assessments, they may be imprecise and, perhaps, over-stated in some cases, as previously discussed. For 
example, program staff consistently reported substantial effort towards collaboration between teachers 
and LIT’, and frequent use of small group instructional settings. However, deeper understanding of the 
implementation of the targeted intervention, as obtained through interviews with district and school 
leaders and through the case study, reveal that there remained important aspects of the model—most 
notably, the authentic use of differentiated instruction—that were implemented inconsistently. Similarly, 
even though the intensive intervention model received generally high fidelity ratings in most sub- 
components, there were indications that the model may be mote effective for some students than others; 
but more importantly, low overall enrollment rates (42% of eligible Tier 3 students did not even sign up 
for AMP) and frequently low attendance rates severely limited the amount of exposure that Tier 3 
students received. 


A major factor that is directly relevant to variations in fidelity of program implementation is staff 
turnover. It was noted above (Banilower ef a/, op. cit; Sarason, op. cit.) that full institutionalization of 
systemic change can take several years. However, this process is set back even further every time a 
trained teacher is replaced by a new one, and teacher turnover is a significant problem in Chicago. 
Although CPS does not maintain regular data on turnover rates, a study conducted by the Consortium 
on Chicago School Research (Allensworth ef a/,, 2009) underscored the severity of the problem. 
According to the study, 100 Chicago schools suffered at that time from chronically high turnover rates, 
losing a quarter or more of their teaching staff every year. In the last year of the study (school year 2006- 
2007), 12 of the 31 Striving Readers schools had lost at least 24% of their teachers since the previous 
year, including eight that lost at least a third of their teachers in just one year. If similar turnover rates 
were to continue throughout the duration of the Striving Readers program, these 12 schools could see 
virtually complete turnover of their teaching staff with a year of the program still to go. Even the 
schools with the three highest stability rates that year—holding onto between 91% and 93% of their 
teachers—would have seen a turnover of more than a third of their teaching staff during the course of 
the five-year program. This turnover occurs not only among classroom teachers; there has also been a 
significant amount of turnover among LITs, and to a lesser extent, principals. Such lack of stability 
hinders the establishment of the Striving Readers program not only at the classroom level, but at the 
administrative, management and interpersonal levels as well. 


While limited implementation fidelity can reduce actual program impacts, initiatives at 
control schools designed to improve performance can narrow the gap and make real impacts for 
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treatment students harder to detect. Although Striving Readers principals did perceive their 
schools more positively than control school principals in a number of important respects, 
ptincipals’ perceptions of the success of their schools’ literacy efforts were indistinguishable 
between Striving Readers and control schools in a number of key areas. These included 
purposeful assessment and data-driven instruction, integration of literacy into the content areas, 
and integration of technology into literacy instruction. 


In addition to factors affecting the quality of program implementation, instructional efforts at the 
control schools can also make treatment effects difficult to detect. As pointed out during a presentation 
at the 2009 Research Conference of the U.S. Education Department’s Institute of Education Sciences 
(Buckley e¢ a/, 2009), detectable treatment effects may be reduced not only when imperfect program 
implementation results in a reduction in actual impact, but also because of conditions that result in 
greater than expected impacts at control schools. 


Because of the low survey response rates from control schools, it was not possible to make explicit 
adjustments to the impact estimate to account for this phenomenon. However, there was a considerable 
amount of evidence that the phenomenon exists and would be expected to further reduce the impact 
estimates. As discussed in the Year 2 report, control schools have been (not unexpectedly) undertaking 
their own initiatives to improve literacy instruction, including some that may have been inspired by the 
Striving Readers program. Additional evidence from principal interviews, which were completed for a 
representative majority of both treatment and control schools, indicates that principals from the two 
groups of schools often do not perceive conditions differently, on average. (Detailed comparisons of 
selected principal interview results are presented in Appendix H.) 


The perceptions of treatment and control school principals about their schools were 
indistinguishable in many respects. These included their perceptions of the success of their literacy 
teams with efforts such as addressing the needs of struggling readers, using student assessments to drive 
instruction, and supporting vertical and horizontal teacher collaboration; and their schools’ overall 
success in using assessment data to screen students for intervention programs, assess the success of 
instructional programs, and support differentiated instruction. In addition, there were no detectable 
differences between groups of principals in their perceptions of the extent to which literacy instruction 
was integrated into the content areas, nor in the overall integration of technology into their literacy 
programs. 


Nevertheless, there were a number of important areas in which Striving Readers and control school 
principals did perceive their schools differently, and that would be expected to contribute to measurable 
differences in impact. Among these were the success of their literacy teams at improving their school’s 
literacy instruction, which Striving Readers principals rated higher than control school principals. In 
addition, Striving Readers principals rated their schools more positively in several aspects of 
implementing technology, including the proportion who said they had handheld computers or laptops*¢ 
available to support literacy instruction (100% of Striving Readers principals vs. 52% of control school 
principals); the extent to which handheld computers or laptops were integrated into the literacy 


56 Since Palm Pilots serve a purpose very similar to what a laptop might, but are seen very infrequently at most schools, 
control school principals were asked on these questions about their use of laptops instead of handheld computers. 


100 


curriculum (among those schools that had them, 37% vs. 9% of principals, respectively, felt they were 
very well integrated); and the extent to which media centers were well integrated (63% vs. 27% of 
principals, respectively, felt they were very well integrated). 


It is also important to note that evidence that focuses on the availability of resources, and the 
frequency of use of resources or instructional methods, does not necessarily reflect additional differences 
between Striving Readers and control schools resulting from differences in the specific content, nature 
and quality of these resources and methods. Thus, even to the extent that many of the control schools 
report using technology to support literacy instruction or using text sets to support the integration of 
literacy into the content areas, it is still expected that the quality and alignment of the training and 
specific resources provided through Striving Readers would result in greater impact. Nevertheless, while 
more objective implementation measures might reveal greater distinctions between Striving Readers and 
control schools, the similarities in principals’ perceptions imply that the control schools may be making 
considerable progress in independently improving their own literacy programs. 


Year 4 Implementation Study 


Intervention as Implemented and Comparison of Implementation in Years | to 
4 


In order to calculate scores representing each school’s program implementation fidelity for the Year 4 
study, the rubrics that were used in Year 3 to generate these scores were modified to reflect the updated 
data sources that were used in Year 4 (see Table 21 below). In addition, comments on district-level 
program leader interviews and district-wide principal interviews, as well as results of individual items 
from teacher and LIT surveys, were used to further illuminate some of the results of the fidelity rubrics. 


The fidelity rubric scores for Year 4 were again generated by comparing actual versus intended levels 
of implementation on various factors for the professional development model and the classroom model. 
However, these scales underwent some additional modifications in Year 4. These changes reflected 
modifications to LIT and teacher surveys, which primarily included removing or streamlining some items 
in order to make the surveys less time consuming for respondents. In addition, some of the survey items 
were reorganized so that they were only asked of those staff for whom they were most pertinent. Finally, 
librarians were not surveyed in Year 4 because many schools did not have a librarian on staff. 


Adjustments were made to the fidelity scores rubric that reflected these changes. Some of the major 
fidelity scale revisions included the following: 


e Addition of a new component to assess fidelity of integration of literacy into content area 
instruction.>7 


57 This component could not be created in Year 3 due to low survey response rates among content area teachers. 


101] 


e Removing the sub-component score for use of the gradual release model from the whole-school 
intervention fidelity score, as this practice is more applicable to the targeted and intensive 
interventions than the whole school, blended intervention. (Items addressing gradual release are 
now incorporated into the targeted and intensive intervention components.) 


¢ Modifying scales to reflect new or modified items that clarified the use of instructional strategies 
and practices, materials, resources, and assessment data specific to the blended, targeted, and 
intensive interventions. 


e Modifying scales to reflect new or modified items regarding the school library and teacher- 
librarian interactions, text sets, other non-technology resources and materials, and the use of 
handheld computers. 


e Adding items to the sub-component score for targeted and intensive professional development 
to assess the perceptions of LIT's about the quality and usefulness of the Striving Readers 
professional development for these interventions. 


As in Year 2, fidelity scores were only calculated at the school level. All scores except the 
professional development scale were again defined on a 10-point scale, where a 1 indicates that none of 
the key program characteristics were being implemented (according to self-reports on surveys and 
interviews), and a 10 indicates that all key components were being implemented with the expected 
regularity. (The professional development scale starts at 0, which represents a low average rate of 
attendance at all of the key training activities.) As discussed for the Year 2 study, it should again be 
noted that a 10 does not represent a “perfect” score. Implementation levels for fidelity scores were 
interpreted as in Year 3: scores ranging from 8 to 10 represent igh implementation (H), scores above 5 but 
lower than 8 represent medium implementation (M), and scores of 5 or lower represent /ow implementation (L). 


Table 21 below summarizes the major changes in the fidelity scales from Year 3 to Year 4. For a 
complete list of the Year 4 LIS survey and interview items used in the creation of scales for each 
component and sub-component, and the formulas indicating the weights given to each item and 
responses to each item, see Appendix I. For a detailed comparison of the changes in the scales, these 
Year 4 scale definitions can be compared to those for Year 3, which are provided in Appendix E. 
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Table 21: Changes in Fidelity Scales from Year 2 to Year 4 


Components and Sub-components 


Yr 4 | 


Changes from Year 2 to Year 4 


Component |: Whole-School al <4 J 
(Blended) Intervention Year 2-Year 3: 

_posurrereerreererrininorerrerrereerrnnirreierereereereerrcoaeeeeeecccoo Changed sub-component | from whole-part-whole to small- 
Sub-component |: Whole-part-whole (Year group instruction due to limitations in the ability to assess 
2)/ Small-group instruction (Year 3)/ J J whole-part-whole. ™! 

Individual and small-group instruction (Year 
4 

bee [Seinnieimninminnmninimnmnnmnomnomnon ——,_——— Refined/added/removed survey items assessing sub- 

J J components 2-5 (e.g., moved items about text sets to 
Sub-Component 2: Gradual release model = materials, excluded vocabulary items from observations). 
Sub-Component 3: Comprehension focus V V V Year 3-Year 4: 

Eliminated sub-component 2 and moved items relating to 

i Sub-Component 4: Use of PRC2 —_—_—_—_—_—IEReOT gradual release from the whole-school model to the targeted 
; ‘ : and intensive models. 
instructional frameworks, text sets, and J al J 
technology to support differentiated 
instruction Refined/added LIS survey items assessing sub-components 3 

hin hl and 5 (eg. seading- comprehension tein, yocbulacy 
Sub-Component 5: Marzano’s Vocabulary | \ V V instruction). 

Year 2-Year 3: 

Component 2: Targeted V 1 1 Added survey items about the nature of teacher/LIT 
Intervention collaboration. 

Added items in the LIT survey about the use of practices to 
ee eecae V V V help students improve comprehension, vocabulary, and 
covaporanon fluency during the targeted intervention. 

Sub-Component 7: Small-group setting for aka ahi 

Tier 2-3 students with direct instruction in | | | Refined/added LIT survey items assessing sub-component 7 

comprehension, vocabulary, and fluency (e.g., differentiated instruction, gradual release model, reading 
comprehension, and vocabulary instruction). 

Component 3: Intensive y ‘ V Year 2-Year 3: 

Intervention Incorporated Tier 3 sixth graders who were not enrolled in 

Sub-Component 8: Increased instructional ad J “f ee eRe Pen ERC ee ee eae: 

time 

ee TTT... 0.00 pereenesanenedaneessnensnnense Added items in the LIT/AMP survey to assess the use of 
Sub-Component 9: Small-group setting V V V strategies and techniques during the intensive intervention. 
Sub-Component 10: Direct instruction in mT a Added sub-components 10 through 12 to reflect new survey 
comprehension items on issues not previously addressed. 

Sub-Component | |: Direct instruction in y V 
vocabulary Year 3—Year 4: 
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Components and Sub-components 


ee 


yea |] vr 4 | 


Changes from Year 2 to Year 4 


Refined/added LIT survey items assessing sub-components 10 


Sub-Comp erent U2 Direct isan ale V V through 12 (e.g., gradual release model, reading 
guided fluency practice comprehension, and vocabulary instruction). 
Component 4: Purposeful Year 2—-Year 3: 
Assessment and Data-Driven v v v Added sub-components about fidelity of use of assessments 
Instruction by intervention type. 
Sub-Component 13: Whole-school | | Added principal interview items about the school’s use of 
(blended) intervention assessment data for a variety of purposes, as well as principal 
IF lll ce ee ratings about the quality of the literacy and grade-level teams 
in using assessment data, under sub-component 13. 
Sub-Component | 4a: Targeted intervention V 
nn, VV Added LIT/AMP sub-component 14 to reflect new survey 
items about the use of assessment data during the after- 
school program. 
Sub-Component | 4b: Intensive intervention V V 
Year 3—Year 4: 
Added targeted intervention sub-component 
Year 2-Year 3: 
Component 5: Materials V V V Reorganized items to create sub-components for each type 
nn, of material and resources used. 
Sub-Component | 5: Text sets V V 


Refined/added/removed LIS survey items assessing sub- 


.| components I5 through 20 (e.g., removed items from the 


pre-observation checklists and the district-wide observations, 


Sub-Component 1 6: School library V V added teacher and LIT survey items about the use of 
ee pandheld cormpater and.school libyati-s): 
Sub-Component | 7: Classroom library V V Venn d Vespa: 
ee ie ore RI | ee ee Eliminated ELA teacher survey items regarding text sets from 
Sub-Component 18: Other non-technology J sub-component |5, as text set use is more applicable to 
resources/materials V content-area instruction. This sub-component is now based 
nn | only on principal interview items about content teachers’ use. 
Sub component 17: andnelt eo espuies V V Refined/added/removed LIS survey items assessing sub- 
TTT components |6 through 19 (e.g., removed items about the 
school librarian because many schools had no librarian, 
k expanded the number of non-technology resources and 
ee Bren L 20 SO Hier teu inaloey V V materials in the materials section of the teacher and LIT 
surveys, and added LIT survey items about using handheld 
computers for differentiation). 
Component 6: Integration of This component was calculated for the first time in 2009-10 
Literacy into Content Area V based on responses to the LIS survey from teachers 


Instruction 


providing content area instruction. 
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Components and Sub-components r Yr3 || Yr 4) Changes from Year 2 to Year 4 


Year 2—Year 3: 


V v Disaggregated PD component into separate sub-components 
for the whole-school intervention vs. the targeted and 
re intensive interventions. 


Added teacher survey items and principal interview items 
about staff perceptions of the usefulness of PD they 
participated in. 


J al Year 3—Year 4: 

Added items to assess the perceptions of LITs about the 
quality and usefulness of the Striving Readers professional 
development. 


ll Although this component was not specifically assessed, it was still expected that teachers follow this model. 


The following discussion of implementation of the Chicago Striving Readers program during the 
2009-2010 school year is organized around the seven main components of the program model. Within 
each component, more specific findings related to the sub-components of the model, as well as to other 
topics that are important to the program but are not explicit model components, or that apply across 
more than one component, are also discussed. Results of the fidelity scales for Year 4 program 
implementation are presented in Figures 10 through 25 (scale results by school are presented in 
Appendix J). These results are discussed below along with findings from additional survey items and 
interviews with program leadership that help illuminate the fidelity scale results. In addition, findings 
from case studies that were conducted during the school year at six of the higher implementing schools 
are also discussed at the end of this section. 


Overall Fidelity5® 


All schools in both cohorts were implementing the classroom model at least at a 
medium level of overall fidelity during school year 2009-2010, including several that 
were implementing with high overall fidelity, showing progress over Years 2 and 3. 
Increased collaboration between project leadership and District Coordinators and 
improved alignment between SR and other district priorities have helped strengthen 
implementation, but substantial challenges remain. 


Results for overall fidelity scores for the overall classroom model are presented in Figure 10. 


58’To maintain comparability to prior years, Component 6 for integration of fidelity into content area instruction was not 
included in the overall fidelity score. 
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Figure 10: Results of Years 2 through 4 Implementation Fidelity Scales 
Classroom Model - Overall Fidelity 
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Overall Fidelity 


As the overall fidelity scores in Figure 10 above and by school in Appendix J show, all schools in 
both cohorts were implementing the classroom model of the Striving Readers program at least at a 
medium level of overall fidelity during the 2009-2010 program year. Reflecting an improvement over 
Years 2 and 3, this included eight schools (32"%)—including two schools from Cohort 1 and three from 
Cohort 2—that were implementing at a high level of fidelity. 


Evidence of similarities and differences between Striving Readers and control schools also indicate 
that, at least in some areas, the program appears to have had a meaningful impact on the implementation 
and delivery of literacy instruction in the schools. For example, according to the Year 4 principal 
interviews, although similar proportions of Striving Readers and control schools (92% vs. 96%, 
respectively) had grade level teams providing a horizontal focus on instruction across subject areas, 
somewhat greater proportions of Striving Readers schools (89%, compared with 78%) had a vertically 
organized literacy team focusing on literacy issues across grade levels. 


The composition and schedules of these teams were also somewhat different for the two groups. 
Among those schools that had such teams, all literacy teams at Striving Readers schools included the LIT 
and almost all (96%) included a special education teacher, compared to less than half (47%) of control 
school teams that had a reading specialist and only about four fifths (82%) that had special education 
teachers. In addition, literacy teams at many Striving Readers schools met more frequently than control 
school teams: the large majority of the former (86%) met at least biweekly--including two that met 
several times a week—compared with only 61% of control school teams that met this often. For grade 
level teams, while their schedules were comparable, team compositions again differed in some ways. 
Almost all Striving Readers teams included their principal as a member (92%) as well as the LIT 92%), 
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compared to 77% and 46% of control teams, respectively, that included the principal or a reading 
specialist. Other differences pointed to the efforts of the Striving Readers grade level teams to address 
all student needs: compared with control schools’ teams, larger proportions included a special education 
teacher (87% vs. 77%) and an ESL teacher (46% vs. 32%). In contrast, a smaller proportion of grade 
level teams at Striving Readers schools than control schools included content area teachers (62% vs. 
86%, respectively). However, the latter result is difficult to interpret: since most teachers teach more than 
one subject, it may reflect principals interpreting ELA as their grade level team members’ primary subject, 
rather than a lack of representation of other subjects on the team. 


Striving Readers principals also seemed to have somewhat more confidence in the effectiveness of 
both their literacy and grade level teams, at least in relation to literacy instruction: although they were 
slightly less positive about these teams’ performance in “addressing the needs of all students,” Striving 
Readers principals were somewhat mote likely to give high ratings to their literacy teams’ performance in 
“improving literacy instruction at your school” (56% rated it as excellent, compared to 44% of control 
school principals). More notably, their confidence in their grade level teams might reflect a greater focus 
on literacy among many Striving Readers schools: over half of Striving Readers principals (54%) rated 
grade level teams’ performance in improving literacy instruction as excellent, compared to less than a 
fourth (23%) of control school principals. Other differences in Striving Readers and control school 
principals’ perceptions of their literacy programs are described in the discussions of the various program 
components, below. 


A number of factors have been cited by program staff at the school and district levels as having 
helped support implementation. The Project Director has continued her close collaboration with the 
District Coordinators to develop their skills and establish priorities. Further support was provided by 
the senior literacy advisor, who also began to work directly with the Coordinators as of Year 4. This 
collaboration helped to further strengthen the Coordinators’ roles, while providing the literacy advisor 
with more direct input into project implementation. Perhaps as a result of this collaboration, District 
Coordinators also reported having fewer responsibilities extending beyond the SR program during Year 
4, which helped them to focus on Striving Readers. In addition, the literacy advisor continued to work 
closely with SR school principals, helping to foster within-school accountability. 


Since the staffs sense of priorities and accountability is largely molded by that of the principal, it is 
clear that the principal’s commitment to and involvement in the program ate critical, and these 
characteristics have been frequently cited by district staff as factors in some schools’ success—and as a 
challenge for others. Principals in turn have cited the availability of the LIT as a strength of the 
program, as well as the program’s positive impact on collaboration amongst teachers. This collaboration 
is strengthened by the model’s instructional consistency across classrooms and by teachers’ shared 
knowledge base and pedagogical vocabulary, even while it is challenged by difficulty finding the time 
needed for active collaboration. 


As has been discussed in relation to prior years’ findings, various CPS initiatives to strengthen 
literacy instruction have been taking place district-wide, including at the control schools. And to some 
extent at least, the Striving Readers program may be influencing these initiatives. (For example, district 
level professional development efforts around literacy and differentiated instruction have been informed 
by the work of ORLA, and by the Striving Readers training model in particular.) This strengthening of 
control programs and alignment of focus may have resulted in further narrowing some of the differences 
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between Striving Readers and control schools’ literacy programs. For example, based on principal 
interviews, differences between the two groups in their use of student data for grouping or to assess 
instructional programs, and in their integration of literacy into math and science curricula, were almost 
indistinguishable. Many other aspects of control schools’ literacy curricula are probably also more 
similar to Striving Readers as a result of the alignment in priorities. While this is a positive outcome for 
both Striving Readers and control schools, it might further obscure the evidence of impact of the Striving 
Readers program, as discussed further under Implications for Impact Analyses, below. 


While Chicago Striving Readers enjoyed greater success in Year 4, it still has to compete with other 
district requirements; the Project Director and the rest of the district SR team can work to establish 
strong relationships with schools and to convince them of the value of the model, but cannot mandate 
participation. Despite the improvements discussed above, the literacy advisor has noted a “lack of 
energy” around implementation. In spite of the positive message that might be inferred from the fact 
that district training efforts have been informed by Striving Readers, conflicting signals about priorities 
and expectations remain. For example, in some cases Chief Area Officers®? have required their schools 
to shift the focus of their LITs more toward coaching than interventionist. Schools with higher 
achievement levels that become Autonomous Management and Performance schools (which retain 
autonomy from Area management) tended to struggle less with implementation. These mixed signals— 
combined with the difficulties that the evaluation has encountered in identifying measurable impact 
relative to control schools—are likely one reason for the indifference in some schools. As discussed in 
prior years, school size also remains an additional confounding factor, as the LIT’s time is spread out 
more in larger schools. (Nevertheless, even some larger schools have been able to achieve relative 
success in their implementation of Striving Readers, as illustrated by the case studies, discussed below.) 
Other obstacles to program implementation that have been cited by Striving Readers principals— 
perennial challenges that affect all instructional initiatives—include high student and staff mobility, large 
class sizes, and the absence for many students of an academically supportive home environment. In 
addition, the Senior Literacy Advisor has observed that student behavior problems sometimes makes 
teachers reluctant to take on small-group activities. 


Additional insights about the implementation of the Striving Readers model are discussed below in 
relation to each component of the model. Since each component and sub-component was measured on 
the same basis for all schools and classrooms, the results should provide a reliable indicator of the range 
of implementation among schools (or between cohorts) within the same component. 


5° Each Chief Area Officer is responsible for ensuring school improvement and specified outcomes for a geographic 
cluster of schools. 
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Component |: Whole-School (Blended) Intervention 


Most schools implemented the blended intervention model at a medium level of 
fidelity. The strongest implementation was in teaching reading comprehension 
strategies, but small-group and differentiated instruction, and specific instructional 
frameworks and techniques, continued to pose a challenge. 


Fidelity scale results for the whole-school intervention are presented in Figure 11, overall and by 
cohort for Year 4, as well as for Years 2 through 4 overall. 


Figure | 1: Results of Years 2 through 4 Implementation Fidelity Scales 
Component I: Whole School (Blended) Intervention 
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Component 1: Whole School Blended Intervention 


At most schools—82% overall, including 88% of Cohort 1 schools and 73% of Cohort 2 schools— 
the blended intervention model for all students was implemented at a medium level of fidelity. All other 
schools, including over a fourth (27%) of Cohort 2 schools, implemented this model at high fidelity. 
These results also reflect progress since the fidelity scales were first implemented in Year 2: while no 
schools have reported implementing the blended model at a low level of fidelity, the proportion 
implementing at a high fidelity level has nearly doubled, from 10% in Year 2 to 19% in Year 4. 


Fidelity scores were also calculated separately for each of the sub-components of the blended 
intervention model. Figure 12 presents these results. 
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Figure 12: Results of Year 4 Implementation Fidelity Scales by Sub-Component 
Component |: Whole School (Blended) Intervention 
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Component |: Whole School Blended Intervention 


Teaching comprehension strategies. The strongest implementation was on the focus on 
teaching reading comprehension strategies, on which 20 of 27 schools for which ratings were available“ 
(74%) achieved high fidelity). Although different teachers focused on different strategies depending on 
the needs of their students, almost all ELA teachers responding to the spring survey (98%) reported 
teaching one type of comprehension strategy or another on at least a weekly basis, and three fourths 
(75%) did so almost daily. Other techniques that most teachers (58% or more) used multiple times a 
week included establishing the purpose for reading and making connections to background knowledge. 


Small-group and differentiated instruction. Survey respondents were less consistent in the 
use of other techniques: although all Striving Readers schools implemented small-group instruction with 
at least a medium level of fidelity, only three attained high fidelity in this technique. In addition, 
although it was not uncommon for teachers to report using differentiated instruction (94% reported 
doing so at least weekly, and 57% on an almost daily basis), project leadership remained skeptical about 
whether teachers’ understanding of the technique was adequate for their self-ratings to be reliable, and 
District Coordinators noted that more “traditional” teachers were often slower to embrace this 
technique. 


69 Fidelity ratings could not be calculated for two schools that did not return any ELA teacher surveys. 
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Instructional frameworks and techniques. ELA teachers’ self-reports also indicated that their 
use of instructional techniques such as Partner Reading in the Content Area, Too (PRC2 ) and 
Marzano’s vocabulary occurred at a medium level of fidelity in most schools. Cohort 1 schools were 
more successful in the former, with 25% implementing the “use of PRC2, text sets and technology to 
support differentiated instruction” at a high fidelity level (compared with 18% of Cohort 2 schools), 
while Cohort 2 schools were more successful in the implementation of vocabulary instruction and word 
study (including Marzano’s vocabulary, Words Their Way and/or Donald Bear’s Word Study): almost 
half (45%) of Cohort 2 schools reported conducting this instruction with high fidelity, compared to only 
one of 16, or 6%, of Cohort 1 schools. 


Component 2: Targeted Intervention 


Almost three out of four Striving Readers schools implemented the targeted 
intervention with high fidelity. Teachers and LITs formed strong collaborations, and 
new techniques were used to review and strengthen intervention strategies. However, 
many LITs continued to face a challenge to their capacity to serve the whole school. 


Fidelity scale results for the targeted intervention are presented in Figure 13, overall and by cohort 
for Year 4, as well as for Years 2 through 4 overall. 


Figure 13: Results of Years 2 through 4 Implementation Fidelity Scales 
Component 2: Targeted Intervention 
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Component 2: Targeted Intervention 


Teachers at the majority of Striving Readers schools reported implementing the targeted intervention 
at a high level of fidelity: 72% of schools (21 out of 29 across both cohorts) received high fidelity ratings 


for this component. A slightly greater proportion of Cohort 1 schools than Cohort 2 schools (75% vs. 
69%, respectively) reported implementing at high fidelity. These results also reflect some improvement 
over time, as the proportion implementing at high fidelity increased from 65% in Year 3.61 


Principals’ confidence in the effectiveness of their grade level and literacy teams showed similar 
patterns in relation to the teams’ performance in supporting struggling readers as they did regarding their 
overall effectiveness in improving literacy instruction. Specifically, while overwhelmingly positive, 
Striving Readers principals had slightly less enthusiasm than control principals about their literacy teams’ 
ability to support struggling readers (83% vs. 88% rating it as at least good, and 52% vs. 65% rating it as 
excellent). However, they had much greater confidence about their grade level teams’ performance in 
this area, with considerably more Striving Readers principals (42% vs. 23%) rating them as excellent. 


Fidelity scores were also calculated separately for each of the components of the targeted 
intervention. Figure 14 presents these results. 


6! The dramatic increase from Year 2 to Year 3 in the proportions rated at high fidelity can not be assumed to reflect 
program improvement, since there were also substantial changes in the definition of this fidelity scale in Year 3, as 
summarized in Table X above. 
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Figure 14: Results of Year 4 Implementation Fidelity Scales by Sub-Component 
Component 2: Targeted Intervention 


$ 
8 
3 
& 
H 
& 


Cohort | 


Subcomponent 5: Subcomponent 6: 
Teacher/LIT Collaboration Explicit Instruction in Smal! Groups 
N=29 N=29 


Component 2: Targeted Intervention 


Teacher/LIT planning for struggling readers. The strongest aspect of the targeted 
intervention was the collaborative relationships established between teachers and LITs. The majority of 
schools (72%, or 21 out of 29) reported highly positive collaborations between teachers and LITs, a 
necessary dynamic to support effective targeted intervention. The senior literacy advisor has been 
pushing schools to focus more attention on Tier 3 students, and has been working with the district 
coordinators to implement a case study approach to gain more detailed insights into implementation. 
This has involved reviewing individual students’ BRI and spelling inventory scores to obtain a more 
accurate understanding of their abilities than can be obtained from ISAT scores alone, and using these 
findings to explore individualized intervention strategies. Although they reported maintaining tier 
groups, district coordinators did not seem to find the official tiering process particularly beneficial, 
largely because of widely shared concerns about the validity of the ISAT results for struggling readers. 
Nevertheless, the active use of other data sources such as the BRI to assess students’ needs helped 
facilitate instructional planning and progress monitoring. 


At those schools that utilized their LIT's more as coaches than interventionists, as discussed above, it 
would follow that these LITs would have less opportunity to directly support struggling readers, further 
confounding the aforementioned difficulty that LITs at the larger schools faced in serving all those in 
need. In a number of cases, classroom teachers were also found to have acquired an expectation or at 
least preference for their LIT to support them through coaching—a pattern that became apparent, even 
among some of the stronger schools, from the case studies. 
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Another challenge for serving struggling readers was that, according to concerns expressed by several 
principals, the SR model may be less effective for English language learners. (Results of program impact 
analyses provide mixed evidence about principals’ impressions. Hispanic students, many of whom are 
English language learners, responded more positively than other ethnic groups in terms of overall 
program effects for all students; however among Tier 3 struggling readers, non-ELL students responded 
more positively than ELL students. These findings are discussed in more detail in Sections IV and V 
below.) During case study interviews, several teachers shed further light on this phenomenon, explaining 
that it is more difficult to find text sets and other literacy materials in other languages, or that are 
designed specifically for language learners, who tend to require more focus on vocabulary building 
before they can fully participate in other SR instructional techniques. 


Component 3: Intensive Intervention 


All schools reported implementing the intensive intervention at least at a medium 
fidelity level, but implementation was less successful than that of the targeted 
intervention. Among the largest challenges was achieving high rates of enrollment and 
attendance. Schools reported being largely successful in providing instruction in 
comprehension, but were less so in fluency and vocabulary instruction. Many LITs 
questioned the relevance and appropriateness of the AMP program for substantial 
numbers of students. 


Fidelity scale results for the intensive intervention are presented in Figure 15, overall and by cohort 
for Year 4, as well as for Years 2 through 4 overall. 
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Figure |5: Results of Years 2 through 4 Implementation Fidelity Scales 
Component 3: Intensive Intervention 


Component 3: Intensive Intervention 


Implementation of the intensive intervention was less successful than that of the targeted 
intervention. Although all schools reported implementing this component at least at a medium fidelity 
level, fewer than half did so with high fidelity. There was substantial improvement in implementation 
from Year 2 to Year 3, with the proportion of schools implementing at high fidelity increasing from 42% 
to 68%; however, this proportion fell back to 45% during Year 4. 


Some of the challenges that schools encountered are apparent from the analyses of the specific sub- 


components of the intensive intervention. These analyses include summaries of fidelity scores calculated 
separately for each of the components of the intensive intervention, shown in Figure 16. 
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Figure 16: Results of Year 4 Implementation Fidelity Scales by Sub-Component 
Component 3: Intensive Intervention 
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Component 3: Intensive Intervention 


Increased instructional time. As in prior years, one of the most significant challenges of 
implementing the AMP program was providing the amount of increased instructional time called for by 
the model. This requires conducting an after-school program that provides at least the requisite number 
of additional hours of training, a criterion met by virtually all schools. However, instructional time can 
only be supplemented if the targeted students participate in the program. Although enrollment rates 
among eligible Tier 3 students (76% overall®*) were improved compared to prior years, average overall 
attendance rates remained at only 46% across all schools.° 


As was found in previous years, even AMP instructors have acknowledged that the program may not 
be ideal for all students—a phenomenon that has been acknowledged by the senior literacy advisor, and 


62 Unweighted average across schools for eligible sixth grade students. 


63 This figure represents attendance for sixth grade students only. It is calculated by dividing the number of minutes that 
each student attended by the minimum number of minutes that the programs should have been offered according to 
the model, rather than the time that AMP was actually offered at each school. For the purpose of this fidelity score, 
eligible students who did not enroll in AMP were counted as having a 0% attendance rate. 
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that may help explain the relatively low enrollment and attendance rates. On spring 2010 surveys, the 
vast majority of LITs 93%) still felt that the program was at least “somewhat” appropriate (including 
41% who felt it was “appropriate” or “very appropriate”) to the reading levels of the students who were 
participating. However, considerably smaller proportions than last year—48% and 35%, respectively— 
felt that the program was appropriate to the literacy needs and learning styles of “most” participating 
students (see Table 22). 


Table 22: Spring 2010 Literacy Improvement Survey: LITs 
Proportion of Students for Whom You Think the Following Statements About 
AMP Are True 


‘ oul! Most About A few Barely 

The AMP after-school program is... ellanrestar-| any 
students | half students 
students students 

... engaging. 29 10.3% 20.7% 31.0% 20.7% 17.2% 
...relevant to their interests. 29 3.4% 24.1% 27.6% 27.6% 17.2% 
... motivating. 29 6.9% 13.8% 27.6% 27.6% 24.1% 
...appropriate to their literacy 9 24.1% 74.1% 34.5% 6.9% 10.3% 
needs. . : . : : 


...appropriate to their learning 


29 13.8% 20.7% 34.5% 13.8% 17.2% 
style. 


Direct instruction in comprehension, vocabulary, and fluency in a small-group 
setting . The Striving Readers schools were highly successful in providing the AMP courses in a small- 
group setting; indeed, all schools but one met or exceeded the teacher:student ratio required by the 
model. The large majority (21 out of 29, or 72%) also reported providing explicit instruction in 
comprehension with high fidelity. They were less successful, however, in their implementation of the 
other instructional techniques. Only 34% of all schools implemented guided fluency practice during the 
AMP classes with high fidelity, and only 28% did so for explicit instruction in vocabulary. 


Component 4: Purposeful Assessment and Data-Driven Instruction 


Fidelity scale results for purposeful assessment and data-driven instruction are presented in Figure 17 for 
Year 4, overall and by cohort, as well as for Years 2 through 4, overall. 
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Figure 17: Results of Years 2 through 4 Implementation Fidelity Scales 


Component 4: Purposeful Assessment and Data-Driven Instruction 


Component 4: Purposeful Assessment and Data-Driven Instruction 


As shown in Figure 17, almost all the schools (96%) were rated as middle or high implementers in 
their use of assessment data and data-driven instruction (48%, respectively). Only one Cohort 2 school 
was trated low in this area. When looking at changes over time, results show that Striving Readers has 
made substantial progress in this area. For example, the proportion of schools implementing this 
component with high fidelity increased from 10% in Year 2 to 36% in Year 3 and 48% in Year 4. The 
Project Director feels that this increase may be attributable, in part, to a push for more rigorous 
performance and accountability measures from Area Officers and district mandates. Nevertheless, it 
remains unclear whether self-reports of increased use of data necessarily reflect more effective use of data 
to drive instruction. 


Fidelity scores were also calculated for purposeful assessment specific to the whole school blended 
intervention and the targeted and intensive interventions. Figure 18 presents these results. 
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Figure 18: Results of Year 4 Implementation Fidelity Scales by Sub-Component 
Component 4: Purposeful Assessment and Data-Driven Instruction 
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Component 4: Purposeful Assessment and Data-Driven Instruction 


Results seem to indicate greater use of assessment data for struggling readers (through the targeted 
and intensive interventions) than for the whole school blended intervention. Specifically, 13 schools 
(45%) reported high levels of fidelity in the use of data for the targeted and intensive interventions as 
compared to 8 schools (31%) for the whole school intervention. On the other end of the spectrum, 4 
schools were rated as low implementers for the whole school blended intervention compared to only 
one school for the targeted and intensive interventions. Cohort 1 schools had a slightly higher level of 
fidelity for the targeted and intensive interventions than Cohort 2 schools. When comparing the targeted 
and intensive interventions to each other, data presented in Figure 18 show that schools had higher 
levels of implementation fidelity for the targeted intervention than the intensive intervention (59% of 
schools were high implementing compared to 38%, respectively). 


Results from principal interviews suggest that Striving Readers schools have made significant 
progress in this area and in some instances are outperforming their control counterparts. For example, a 
considerably higher proportion of principals in treatment schools than in control schools rated the 
quality of the literacy team’s performance in using assessment data and/or student work to drive 
instruction as “excellent” (52% and 28%, respectively). Furthermore, more principals in treatment than 
control schools also rated the quality of grade-level teams in using assessment data to plan for instruction 
as “excellent” (50% compared to 18%) and using assessment data to establish vertical and horizontal 
literacy goals by grade level as “good” or “excellent” (88% compared to 68%). 


There was also additional evidence that Striving Readers schools have continued to integrate the use 
of assessment data into their school culture. For example, the large majority of school principals 
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reported using student assessment data for a variety purposes to a “moderate” or “large extent.” 
Purposes included screening students’ ability levels for placement in intervention programs (96% 
reported using data for this purpose to a moderate or large extent), diagnosing students’ strengths and 
support needs for placement in specific courses or instructional groups (96%), identifying trends in 
fluency and comprehension abilities across groups of students (100%), identifying trends in vocabulary 
knowledge across groups of students (85%), monitoring overall student progress for the purpose of 
assessing success of instructional programs or methods (100%), differentiating instruction (92%) and 
planning on-site professional development (100%). Furthermore, significant differences were observed 
between treatment and control schools in their use of assessment data to screen students’ ability levels 
for placement in intervention programs, with a much larger proportion of treatment principals than 
control principals reporting that they use data for this purpose to a “large extent” (77% compared to 
44%, respectively). Table 5 in Appendix K shows the detailed results. 


In the surveys, literacy teachers also reported high levels of use of assessment data. For example, 
overt 90% of teachers indicated using data from a variety of assessments, including the Reading 
Benchmark Assessment (94%), the ISAT (98%), the BRI (90%), informal assessments (98%), fluency 
snapshots (98%), and spelling inventories (97%), among others. Furthermore, a majority of teachers 
indicated that they use data from these assessments to a “moderate” or “large extent” for a variety of 
purposes, including placing students in intervention programs (68%), differentiating instruction (92%), 
identifying skills that need to be taught or retaught (93%), monitoring students’ progress (94%) and 
creating instructional groups in class (89%). 


Component 5: Materials 

Fidelity scale results for integrating high-quality, high-interest materials into the literacy curriculum and 
instruction are presented in Figure 19 for Year 4, overall and by cohort, and for Years 2 through 4, 
overall. 
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Figure 19: Results of Years 2 through 4 Implementation Fidelity Scales 
Component 5: Materials 
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Component 5: Materials 


Results show that, in Year 4, the “high interest, high quality materials” component of Striving 
Readers was implemented at a medium level of fidelity in a majority of schools (88%). Cohort 1 schools 
showed a slightly stronger level of implementation than Cohort 2 schools. When comparing results 
across years, data show that, in Year 4, there was some progtess in the area of integration of high 
interest, high quality materials into the literacy curricula. For example, although the proportion of 
schools who implemented this component at a medium level remained constant from Year 3 to Year 4 
(87% and 88%, respectively), in Year 4, the remaining schools scored at a high level of implementation 
(12%), whereas in Year 3, the remaining schools scored at a low level of implementation (13%). 


Fidelity scores were also calculated for each of the main types of materials and resources emphasized 
in the Striving Readers model, including technology resources (i.e., handheld computers, media centers, 
and listening centers) and non-technology resources (i.e., text sets, classroom libraries, school library, and 
other). Each of these is described in more detail in the following paragraphs, complemented by findings 
from other sources such as the teacher surveys, principal interviews and district staff interviews. 


Technology Resources. Technology integration is an important component emphasized by the 
Striving Readers model. Treatment schools have been provided with several technology resources, 
including handheld computers, media centers and listening centers, and teachers have received ongoing 
professional development on the use of these resources. 


Results from the principal interviews were very positive and suggest that Striving Readers has had a 
strong impact on technology use. For example, significant differences were found between treatment and 
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control schools, with treatment schools reporting higher levels of integration of overall technology, as 
well as specific technology resources, into the literacy curriculum. As shown in Table 23 below, a much 
larger proportion of principals in treatment schools than control schools reported that technology has 
been thoroughly integrated into the literacy curriculum (54% compared to 18%, respectively), including 
media centers (58% compared to 10%), listening centers (39% compared to 10%) and 
handhelds/laptops (50% compared to 9%). 


Table 23: Integration of Technology Resources Into Literacy Curricula 


To what extent 
have the following 
iu-Yel lake) (e347 
resources been 


Treatment Schools (N = 26) Control Schools (N = 21, 21, 22, 22) 


integrated into NA Not atall § Somewhat Thoroughly NA Not atall © Somewhat Thoroughly 
the literacy integrated integrated _ integrated integrated integrated integrated 
curriculum?* 

3.8% 0% 38.5% 57.7% 19% 0% 71.4% 9.5% 

0% 3.8% 57.7% 38.5% 19% 0% 71.4% 9.5% 

0% 3.8% 46.2% 50% 19% 4.5% 50% 9.1% 

-- 0% 46.2% 53.8% -- 0% 81.8% 18.2% 


* Ana in this column denotes a statistically significant difference (p<.05) in the distribution of responses between 
Treatment and Control groups based on a Mann-Whitney U test. 


Despite these positive findings, additional results from the fidelity analyses suggest that Striving 
Readers schools were still struggling with the implementation and use of both the handheld computers 
and the listening centers—although most of the case study schools were using handheld computers more 
frequently than other schools. Figure 20 below shows the detailed fidelity results for the technology 
subcomponents. 
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Figure 20: Results of Year 4 Implementation Fidelity Scales, by Subcomponent 
Component 5: Materials - Technology Resources 
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Component 5: Materials 


As shown in Figure 20, over 40% of the schools reported a low level of fidelity of implementation 
for the handheld computers, while 52% reported a medium level and 7% reported a high level of 
implementation. This is consistent with survey findings, which show that just over half (55%) of the 
literacy teachers use handheld computers to teach literacy. The main reasons for not using the handheld 
computers included: not participating in sufficient professional development to feel comfortable using 
them (36%), not believing they offer sufficient added benefit compared to traditional media (26%), and 
not receiving the handheld computers (24%). 


Of those teachers who reported using the handheld computers, most indicated using them in a 
variety of grouping strategies, including whole class and/or large group (73%), small groups and/or pairs 
(89%), and individual work (76%). The large majority of teachers reported that students use them less 
than once a week, including 51% who said one to three times a month and 37% who said less than once 
a month. The most common uses of the handheld computers, as reported by at least half of the teachers, 
included: vocabulary development (82%), organizing information (65%), developing students’ self- 
directed learning (62%), teaching word parts (62%) and writing skills (61%), locating information (59%), 
activating students’ prior knowledge (55%), and synthesizing information (55%). Detailed survey 
findings are presented in Appendix L. 


Findings from the technology coordinator interview also suggest that the use of handheld computers 
was mixed, with varying degrees of implementation across schools and within schools. The technology 
coordinator noted that teachers use them in the whole class blended instruction model but not for 
differentiation purposes. The LITs also use them during targeted interventions to provide extra support 
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to Tier 2 and Tier 3 students. However, the coordinator added that there was some confusion among 
LIT’s about whether they were supposed to use the handheld computers during the afterschool program, 
which resulted in a much more limited use of handhelds in that setting. When used, the handheld 
computers were found to be particularly useful in building fluency. Teachers were also able to use them 
to monitor student performance, but according to the coordinator, this did not happen as often as it 
could because the LITs are responsible for overseeing the handheld computers and therefore need to be 
present when in use. Handhelds were considered to substantially increase the motivation and 
engagement of students. 


Use of the listening centers occurred at somewhat higher fidelity than the handhelds, but was still 
problematic. Almost half of treatment schools (46%) reported low levels of implementation, 32% 
reported medium levels, and 21% reported high levels. Results show better fidelity of implementation of 
the media centers, with 31% of the schools reporting high fidelity levels, 62% reporting medium levels 
and 7% reporting low levels of fidelity. According to teacher survey results, both listening and media 
centers were used mostly for differentiated instruction and small group and individual work, but media 
centers were used more frequently than listening centers. Specifically, 73% of teachers reported using the 
media centers at least once a week, compared to 38% for the listening centers. The most common uses 
of the media centers, as reported by at least half of the teachers, included: developing students’ self- 
directed learning (70%), teaching reading comprehension (68%), vocabulary (54%) and writing skills 
(54%), supplementing students’ textbook readings (59%) and activating students’ prior knowledge (55%). 
As for the listening centers, the most common uses were teaching reading comprehension (71%) and 
fluency (60%) and supplementing students’ textbook readings (61%). 


Information about the integration of technology was also obtained through the district staff 
interviews. According to the technology coordinator, the district’s support of technology has changed 
over the last two years, with the position of technology consultant phasing out and the role of the 
technology coordinator solidifying as a position with a strong focus on supporting schools in their 
integration of technology into the curriculum and use within the classroom. This was also possible 
because many of the technical difficulties that schools had experienced had been solved in previous 
years. In Year 4, the technology coordinator visited the schools on average once a month or more often 
when needed. During school visits, the coordinator attended literacy team meetings, met with the 
principals and teachers, and observed classrooms. The focus of these visits was to help schools devise 
ways in which technology could forward their literacy goals. Within this context, the technology 
coordinator reported focusing mostly on the classrooms where the LIT also pushed in. 


Non-technology resources. Figure 21 below shows the detailed fidelity results for the non- 
technology subcomponents. 
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Figure 21: Results of Year 4 Implementation Fidelity Scales, by Subcomponent 
Component 5: Materials - Non-Technology Resources 
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Component 5: Materials 


As shown in Figure 21, classroom libraries had the highest implementation fidelity ratings among all 
non-technology resources. Specifically, 92% of schools (including all Cohort 1 schools and all but one 
Cohort 2 schools) reported high levels of implementation. In the surveys, almost all literacy teachers 
(92%) reported using this resource at least on a weekly basis, including 60% of teachers who reported 
using it four to five times a week. Most of the teachers also reported being “very comfortable” using this 
resource and indicated using it for multiple purposes, including: teaching reading comprehension (83%), 
helping students’ self-directed learning (70%), supplementing students’ textbook reading (68%), fluency 
(62%), vocabulary (60%), teaching content themes (55%), and activating students’ prior knowledge 
(53%). All teachers who use their classroom libraries also mentioned using them to support grouping 
strategies and/or differentiated instruction. 


Over 90% of these teachers also indicated that they use interest inventories to help students self- 
select reading materials and to guide their purchases for the classroom. When selecting classroom 
reading materials, almost all teachers reported considering both the students’ needs and reading abilities 
and their interests and motivation. Teachers were highly satisfied with their classroom libraries, with 
most of them (between 60% and 88%) reporting that the following statements are “very true”: classroom 
libraries are easily accessible to students, well organized and in good shape, representing a variety of 
reading materials that are appropriate for readers of different abilities and readers with different interests, 
include both fiction and nonfiction materials, are grouped by genre, are clearly labeled and have a 
checkout system in place. Detailed survey results are presented in Appendix L. 
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On the other hand, the school library had the lowest fidelity ratings among all of the high quality, 
high interest materials. As shown in Figure 21, over half of the schools reported a low level of 
implementation in this area, 41% had medium levels and 7% had high levels; although Cohort 1 schools 
implemented with higher fidelity than Cohort 2 schools. According to survey results, the majority of 
literacy teachers take their class to the library at least once a week or more (55%). However, almost one 
third (32%) of teachers said they never or rarely bring their class to the library. In Year 4 of the project, a 
total of 17 Striving Readers schools had a librarian on staff. In schools with a librarian, the large majority 
of teachers reported that the librarian had worked with them and their students in a variety of ways, 
including providing resources for class projects, collaborating on how to supplement lessons with library 
resources, working with students on research skills, directing students to resources tied to the 
curriculum, and assisting students with class projects, among others. 


The use of text sets received very high fidelity ratings, with all Cohort 1 schools and nine of the 11 
Cohort 2 schools reporting a high level of implementation in this area. The remaining two Cohort 2 
schools reported low levels of implementation. According to the principals, Striving Readers text sets are 
being used in Social Studies classrooms in all 26 schools where the principal was interviewed, and they 
are also being used in Science classrooms in 23 of the schools. This was somewhat consistent with 
teacher survey results. Specifically, over three-quarters (77%) of content area teachers indicated that they 
use the text sets provided by Striving Readers. Teachers who reported using the text sets had very 
positive views about the usefulness of this resource. For example, most teachers thought that, for a 
majority of students, the Striving Readers text sets were appropriate to their reading levels and their 
literacy needs, were relevant to their interests, were appropriate to their learning styles and had motivated 
them to learn more about a specific topic. At schools where they were not being used, the most 
commonly cited reason was that content teachers felt that they were not sufficiently aligned to their 
curricula. Detailed results are presented in Appendix L. 


Finally, according to fidelity data, almost all (89%) schools implemented other non-technology 
resources at a medium level of fidelity. Specifically, a large percentage of literacy teachers responding to 
the surveys indicated using these other non-technology resources at least once a week, including 
vocabulary notebooks (60%), reading response notebooks (57%), reading anthologies (32%), reading 
basals (50%) and other informational texts (71%). Most teachers also reported using all these materials 
for a variety of purposes, such as teaching reading comprehension, vocabulary and fluency, and to 
support grouping strategies and/or differentiated instruction. Detailed survey results are presented in 


Appendix L. 


Component 6: Content Area Literacy Instruction 


Fidelity scale results for integration of literacy instruction into the content areas are presented in Figure 
22 for Year 4, overall and by cohort. Comparisons across years were not possible due to low response 
rates to content area teacher surveys in previous years. 
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Figure 22: Results of Year 4 Implementation Fidelity Scales 
Component 6: Content Area Literacy Instruction 
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Component 6: Content Area Literacy Instruction 


As shown in Figure 22, all Striving Readers schools implemented this component with medium (55%) or 
high fidelity (45%). Cohort 2 schools reported a higher degree of implementation than cohort 1 schools, 
with, 69% of Cohort 2 schools reporting high fidelity, compared to 25% of Cohort 1 schools. 


Content area literacy instruction has been taking place mostly in social studies and science 
classrooms and to a lesser extent in math instruction. For example, the majority of principals who were 
interviewed reported that literacy has been integrated “to a large extent” in social studies (73%) and 
science (58%), while less than one-third (31%) reported the same for math. Particularly in social studies, 
these proportions exceeded those of the control schools, where only 56% of principals reported that 
literacy was integrated “to a large extent.” According to their survey results, over half (62%) of the 
content area teachers at Striving Readers schools had already started integrating literacy into their content 
area instruction before Striving Readers began. Nevertheless, not all teachers were so open to this 
process, and in fact District Coordinators as well as some principals reported that getting content area 
teachers to acknowledge the importance of incorporating literacy was a significant challenge in some 
program schools. 


Fidelity scores were also calculated for each subcomponent, including comprehension, vocabulary, 


the use of PRC2, interaction with literacy experts and the use of text sets. Figure 23 presents these 
results. 
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Figure 23: Results of Year 4 Implementation Fidelity Scales by Sub-Component 
Component 6: Content Area Literacy Instruction 
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Component 6: Content Area Literacy Instruction 


As shown in Figure 23, fidelity results show that all or almost all schools scored at a medium or high 
level of fidelity in their knowledge and use of comprehension, vocabulary strategies and skills and PRC2. 
For example, in the area of reading comprehension, over three quarters of the content area teachers 
reported doing the following at least once a week: helping students make connections to background 
knowledge (96%), monitoring students’ comprehension through questioning (95%), using differentiated 
instruction (91%), establishing the purpose for reading (89%), explicit instruction in a variety of 
comprehension strategies (87%), synthesizing information within or across texts (84%), making 
connections between texts (82%), and using before, during and after reading strategies (78%). Over half 
(52%) also reported using PRC2 strategies at least weekly. In the area of vocabulary knowledge, a large 
proportion of teachers reported using explicit instruction in vocabulary (92%), reviewing vocabulary 
words (89%), modeling the use of word parts (70%), using academic vocabulary for content terms (e.g., 
Marzano) (66%), using before, during and after strategies (64%), and using vocabulary notebooks (62%) 
at least once a week. Smaller percentages reported using PRC2 (37%), word study sorts and concepts 
(35%), Words Their Way (34%), and morphology instruction (30%). 


In addition, most teachers reported using specific techniques, such as KWL, Guided Reading and 
Summarizing Procedure and ABC Graffiti, to help students develop their reading strategies and skills. 
Other techniques (e.g., ReQuest, Everybody Reads To, Predict-Locate-Add-Note or Interactive 
Notation System for Effective Reading and Thinking) were used less often. Detailed survey results are 
presented in Appendix L. 


All Striving Readers schools had at least medium fidelity ratings (including 34% with high ratings) for 
content area teachers’ interactions with “literacy experts” such as the LIT, literacy team members, 
literacy coaches, district staff; or academic or other consultants. In the survey, a majority of content 
teachers indicated talking at least once a month with literacy experts about the following topics: using 
student assessment data for instructional planning (87%), differentiated instruction (87%), using specific 
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Striving Readers instructional techniques for comprehension instruction (83%), student groupings (82%), 
using specific Striving Readers instructional techniques for vocabulary instruction (81%), discussing 
specific students’ reading progress (78%), using Striving Readers text sets and teacher guides (74%), 
using technology resources (73%), and using the PRC2 instructional framework (62%). 


Finally, fidelity analyses also revealed medium to high fidelity across schools in their use of Striving 
Readers text sets. As mentioned previously, about 77% of content area teachers responding to the survey 
indicated using text sets in their content area. In general, teachers felt these text sets were appropriate for 
their students, and helped them be more motivated and want to learn more about the topics (see 
Component 5 for more detailed results). 


Component 7: Professional Development 


As described in more detail in Table 3b above, Striving Readers continued to offer a comprehensive 
professional development program throughout the fourth program year. The focus of these activities 
included training in Striving Readers concepts and techniques (including Palm Pilots and other 
technologies) for teachers at all grades and all content areas, meetings between LITs and District 
Coordinators on a roughly bi-weekly basis, and building principals’ capacity in literacy and program 
components through bi-monthly sessions. New this year, professional development also included an 18 
hour workshop on special topics in school libraries. A detailed professional development schedule is 


presented in Appendix M. 


Fidelity scale results for professional development are presented in Figure 24 for Year 4, overall and 
by cohort, as well as for Years 2 through 4, overall. 
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Figure 24: Results of Years 2 through 4 Implementation Fidelity Scales 
Component 7: Professional Development 
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Component 7: Professional Development 


As shown in Figure 24, all the schools implemented the professional development component with 
medium (66%) or high fidelity (35%); however, a much larger percentage of Cohort 1 schools than 
Cohort 2 schools implemented this component with high fidelity (50% compared to 15%, respectively). 
A comparison across years shows that there has been considerable progress in this area. Specifically, the 
percentage of schools with a low level of fidelity of implementation has decreased from 13% in Year 3 to 
none in Year 4, whereas those implementing with high fidelity has also increased from none in Year 3 to 
35% in Year 4. These improvements may have resulted from a combination of improved attendance 
(perhaps because the schedule has become increasingly more manageable and accessible in recent years), 
and/or increased perceptions of the usefulness and impacts of these activities.“ 


Fidelity scores were also calculated separately for professional development activities specific to the 
whole school blended intervention and the targeted and intensive interventions. Figure 25 presents these 
results. 


64 Improvements in fidelity of this component from Year 2 to Year 3 can not be interpreted because of changes in the 
definition of the fidelity scales, as summarized in Table X above. 
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Figure 25: Results of Year 4 Implementation Fidelity Scales by Sub-Component 
Component 7: Professional Development 
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Component 7: Professional Development 


Results suggest a higher degree of fidelity of implementation of the PD component for the targeted 
and intensive interventions than for the whole school model, with 62% compared to 14%, respectively 
scoring high in this area. 


Of all literacy teachers responding to the survey, the majority reported attending the Summer 
Institute (70%) and school-year follow-up institutes (61%), the technology training (75%) and the 
school-based professional development sessions (83%). About half (48%) indicated receiving training in 
LIT/teacher collaboration, but only those teachers with whom the LIT’ push-in are expected to attend 
these sessions. Only one fifth (18%) of literacy teachers received training on the AMP intensive 
intervention program; however, most literacy teachers are not directly involved in this program, and 
those who are not were not expected to attend. When asked to rate the sessions that they did attend, the 
large majority of participants (between 71% and 90%, depending on the training) felt that the trainings 
were moderately or extremely useful. The topics in which teachers most often reported having received 
training through these sessions included: differentiating instruction, building academic vocabulary, using 
student assessments to guide and inform instruction, using the PRC2 model, using the whole-part-whole 
classroom instruction model, and using handheld computers. The large majority of literacy teachers 
reported that these trainings have had a moderate to large impact on their teaching practices. 


Most content area teachers have also attended a variety of professional development opportunities 
on similar topics as the literacy teachers, and the large majority reported using these teaching practices 
(and being comfortable using them) as part of their content area instruction. Detailed results are 


presented in Appendix L. 
Finally, principals were very appreciative of the professional development that they and their schools 


have received from Striving Readers. All principals reported attending the monthly principals meetings 
and most indicated that these opportunities have been moderately (28%) or extremely useful (64%). Two 
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thirds or more have also attended the summer institutes (67%) and school-year follow up institutes 
(83%), as well as the on-site training during literacy team meetings (68%) and the school-based 
professional development (73%). As with the monthly principal meetings, principals overwhelmingly 
rated these opportunities as moderately to extremely useful. 


These results confirm that teachers and principals found the trainings useful—at least among those 
who attended. Nevertheless, while principals cited a high level of literacy expertise amongst their staff 
that was brought about by the Striving Readers professional development program, this perception was 
not universally shared, and there remained factors that likely created impediments to efforts to bring 
staff's knowledge and skills to a level that would ensure full implementation. During the spring 
interview, the senior literacy advisor noted that, in retrospect, she realized that many staff possessed less 
foundational knowledge prior to beginning Striving Readers than the model may have required. As a 
result, a considerable amount of effort had to be devoted to building understanding of what had been 
thought to be more basic concepts such as differentiation. This necessity, in turn, detracted from the 
ptogram’s ability to develop staff skills in specific Striving Readers techniques. Further compounding 
these problems was the fact that staff turnover remains quite high in many Chicago schools: the one year 
average®> teacher turnover rate from school year 2008-2009 to school year 2009-2010 was 55% among all 
Striving Readers schools, ranging from 14% to 82% per school. (The impacts of instability were felt 
among other positions as well, from LITs and principals to district coordinators.) As a result, training 
needed to start from scratch each year in many cases. Even among case study schools, some teachers 
commented that they were uncertain of just what was expected of them from the model. This confusion 
might have been related at least in part to these impediments to the training process; although the 
teachers who made this assertion tended to attribute it to a lack of clear guidelines. 


Case Study Findings 


Case studies of six of the higher-implementing schools revealed similarities in the ways 
these schools facilitated program implementation, as well as variations in the 
challenges they faced. Among the characteristics that were common to these schools 
were an actively engaged and supportive school administration; a culture of 
professional learning that valued and pursued professional development opportunities; 
formal and structured opportunities for collaboration; LITs who served as 
interventionists while also providing staff support; and strong and consistent use of 
student data to inform instruction and planning. 


Preliminary trend analyses imply that the case study schools may also have been 
among the most successful at strengthening students’ literacy skills. At five of six case 
study schools, ISAT scores of the 2009-2010 cohort of eighth graders exceeded those 
of the pre-program cohort, and the magnitude of the mean difference between these 
cohorts was greater at five of the six case study schools than it was for the remaining 
treatment schools combined, which in turn exceeded the magnitude of the mean 
difference between eighth-grade cohorts at the control schools. 


65 Unweighted. 
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In addition to the above fidelity analyses, further insights about SR program implementation were again 
obtained through case studies of six treatment schools that were considered by program leadership to be 
among the higher fidelity programs during the 2009-2010 school year.6° A cross-site summary of the key 
findings from these case studies is presented below (school-specific case study summaries are presented 
in Appendix N°’). In these summaries, results are discussed in terms of multiple respondents if they 
were derived from multiple comments, or from a comment from a school leader describing a pattern 
believed to apply across teachers or classrooms within a school. The experiences of individual 
respondents may also be discussed in cases where they represent an example (or counterexample) of a 


29 6¢ 


pattern. In a few cases, discussions of findings are preceded by a qualifier (e.g., “many,” “consistent,” 
“clearly”). Such phrasing is used in reference to case study results only if the pattern applied to at least 
half of the individuals who were described or who commented about a particular topic.®* The strength 
of a finding is also related to the degree of variation among the descriptions or opinions that inform a 
finding: the less variation, the stronger the finding is. Nevertheless, even when a pattern is described as 
“consistent” within the case study, it still cannot be assumed to be generalizable to all Striving Readers 


schools. 


Implementation of the Striving Readers program during the 2009-2010 academic year was explored 
for each of these six schools through interviews with teachers, principals, and Literacy Intervention 
Teachers; observations of ELA, content area, and AMP class sessions; and review of school-level 
quantitative indicators such as demographics, survey results, and achievement scores. What follows is a 
cross-site summary of those findings, starting with a summary of the major themes about 
implementation that emerged across the six schools, followed by a short description of the major sources 
of variation in implementation among these schools, and ending with a brief summary of literacy gain as 
measured by changes in mean ISAT scores. 


Implementation Themes. Across the six case study schools, certain broad characteristics 
emerged—similar ways in which the Striving Readers grant was fostered and facilitated. While the 
specifics of implementation varied, these higher implementing schools shared five overarching traits: 


e avisibly supportive school administration; 
@ principals, teachers, and staff that valued and pursued professional development opportunities; 
e the use of grade-level and literacy-team meetings as productive means of formal collaboration; 


e = Literacy Intervention Teachers who were able to support teachers while also filling their primary 
roles as interventionists; and 


66 See section X above for details of the process used to select the case study schools. 


67 For one school, an individual case study report was not created so as to protect the confidentiality of respondents who 
preferred not to have their comments associated with their school. However, with the permission of the respondents, 
findings from this school are incorporated into the cross-site summary. 


68 Because the open-ended nature of these interviews and observations does not explicitly prompt respondents to 
comment on particular opinions, patterns that are consistent across even a slight majority of respondents should be 
considered highly noteworthy. 
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e =the integration of data-driven instruction and decision making into the school’s overall literacy 
approach. 


Administrative Support and Leadership 


A key factor identified by respondents in each of the six schools was the commitment and leadership 
shown by each school’s administration. Not only were the principals at these schools found to be very 
knowledgeable about their schools’ literacy activities, they also supported Striving Readers through 
concrete actions. This included, first and foremost, becoming personally involved in the activities of the 
grant rather than delegating responsibility. For example, schools benefited from principals who 
participated in literacy team and grade-level team meetings, engaged in conversations around data, and 
monitored classroom-by-classroom use of techniques and strategies. Principals were also found to take 
the lead in ensuring accountability to the model by directly expressing their expectations of teachers. In 
some cases, this entailed encouraging reluctant teachers to embrace pedagogies that they may not have 
used in the past, such as differentiated instruction and the integration of technology into the classroom. 
Other times it entailed working to make sure that specific teachers received the extra assistance they 
needed to effectively implement Striving Readers strategies. Additionally, teachers cited the benefits of 
school leaders who remained receptive to teacher and staff requests pertaining to literacy instruction, for 
example regarding needed professional development and time for collaboration. Finally, the commitment 
shown by school administrators extended to considering ways in which the Striving Readers approach 
could be supplemented and extended. For some, this meant extrapolating best practices for use in 
content area instruction or to elementary grades. For one administrator, it meant allocating discretionary 
funding towards literacy instruction. 


Strong Use of Professional Development 


At each case study school, teachers pursued and took advantage of a variety of training 
opportunities. Teachers and staff (including the principals) attended formal professional development 
trainings offered by the Striving Readers program and simultaneously organized on-site trainings by 
district staff. These opportunities were sometimes augmented by supplemental, school-led trainings, 
organized by literacy and grade-level teams, the LIT, and/or the school administration. The focuses of 
these supplemental trainings were found to be informed by classroom, grade, and school-level data to 
help ensure that they responded to the areas of greatest need. Participation was also found to be wide- 
ranging, with veteran and new teachers attending sessions, along with both English Language Arts and 
content area teachers. While professional development was generally well received, the case study 
respondents identified ways in which it could be enhanced. Teachers cited the benefits of incorporating 
“hands-on” training and opportunities to practice techniques and strategies, the benefits of balancing 
“Sust-in-time” training opportunities with those that reinforced core Striving Readers tenets, and the 
presentation of concrete models of best-practice implementation. Teachers also consistently requested 
that trainings be differentiated by staff experience and teaching area (content or ELA). 
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Significance_of Formal Collaboration 


Another major theme identified was the high level of formal collaboration taking place among 
teachers and staff. At all six schools, both grade-level and literacy teams met regularly and covered topics 
beyond logistics and lesson planning. A wide range of topics were discussed addressing substantive 
issues such as student groupings and individual student progress monitoring through the sharing of 
observations and assessment data. Additionally, it was found that the grade-level and literacy teams often 
had functionally different but complementary roles at these schools. Literacy teams were focused on 
“set[ting] the tempo for all facets” of literacy instruction at the school, working to hone Striving Readers 
strategies, identifying research-based best practices, and planning professional development; while grade- 
level teams tended to focus on curriculum planning and “assuring that ... [the] curriculum is cohesive 
and there is flow from one grade level to the next.” Grade-level teams also sometimes benefited from a 
clustering approach wherein “vertical collaboration” could take place across grade levels. Team-based 
collaboration also benefited from the broad participation of many different school personnel, including 
the principal, the LIT, and in certain cases specialists brought in based on the challenges faced by the 
schools. For example, at one school, the team meetings became a place for teachers to collaborate with 
literacy staff in tailoring the school’s instructional approach to the substantial limited-English-proficient 
student population. Finally, literacy and grade-level team meetings were identified as useful methods of 
sharing concrete expectations about Striving Readers with the schools and ensuring that all components 
of the model were clearly and consistently understood by everyone involved. 


Importance of the Literacy Intervention Teacher 


The Literacy Intervention Teacher also emerged as a keystone of each school’s implementation. 
Teachers and staff described their LITs as providing valued services above and beyond their primary 
roles. The model defines the LIT’s role primarily as an interventionist for Tiers 2 and 3 students, which 
includes direct instruction as well as supporting lesson planning and conducting analysis of assessment 
data for sixth-grade teachers. In particular, teachers across grades looked to the LIT for in-depth 
coaching and in-class modeling of techniques and strategies. Teachers in the seventh and eighth grades 
looked to the LIT for the services provided to their colleagues in grade 6. To a certain extent, across the 
six case study schools, teachers came to expect the LIT to carry out these added activities, and were 
disappointed when they did not occur or had to be limited to the sixth-grade English Language Arts 
teaching staff (e.g. in larger schools). Unexpectedly, the classroom presence of the LIT was also 
described as being tied to the successful use of technology and the differentiation of instruction, both 
important components of the blended whole-school model. Teachers indicated that having a second 
individual in the room to help monitor the students was essential. Therefore, for a portion of teachers, 
the involvement of the LIT became critical to their adoption of even those aspects of the Striving 
Readers model for which they would normally have had primary responsibility. Overall, these findings 
also suggest that teachers might benefit from additional training on how to independently carry out the 
techniques and strategies of the Striving Readers model. The extent to which teachers rely on their LIT's 
for support beyond their originally intended role might signal that these teachers are not sufficiently 
prepared to carry out their own programmatic roles independently. This could be a result of a program 
and training plan that assumed too much baseline knowledge, as the senior literacy advisor has 
suggested. 
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Integration of Data-driven Instruction and Decision Making 


Teachers and staff at each case study school integrated data collection and analysis into their 
approach to literacy instruction. At the student level, academic and assessment data were used to 
monitor student progress, while teachers reported also incorporating behavioral and socio-emotional 
information to provide a context for better understanding the struggles and successes of each student. In 
addition, each of the six schools used aggregate data for purposes including the selection of professional 
development topics, grade-level curriculum planning, and the tracking of classroom outcomes for the 
purpose of ensuring accountability to the Striving Readers model. Teachers and staff also supplemented 
the data collected through their Striving Readers—prescribed assessments by using additional tests and 
measures to gain information about the literacy skills of their students. For example, some schools used 
independently funded NWEA assessments and some schools applied the results of CPS-mandated 
Scantron assessments to their literacy planning and Striving Readers discussions. Other indicators of an 
attention towards data included schools independently verifying the tiering of students using in-house 
data and the distribution of responsibility for data analysis across multiple staff positions and levels 
(instead of relying solely on the LIT or a specific “data person” to manage the information). 


Variations in Implementation Among Case Study Schools. Apparent throughout the 
findings of the Year 4 evaluation report is that there is a range of ways in which the Striving Readers 
program has been carried out by participating schools. Within this broad context, the case study 
approach taken with the six relatively high-fidelity schools provides an excellent opportunity to explore 
the underlying sources of these between-school differences in implementation. While each case study 
encapsulates the unique unfolding of Striving Readers at that particular school, three influential factors 
consistently appeared: the characteristics of the student population; the characteristics of the school itself 
(including of the teachers and underlying pedagogical approach); and the characteristics of any 
concurrent mandates, programs, and grants. 


Student Characteristics 


Among the student characteristics most commonly cited by teachers and staff as influencing 
program implementation at the case study schools were the percentage of special education and English 
language learning students, the overall degree of homogeneity of the school population—which 
impacted the extent to which teachers could hold the interests of all students with the same materials and 
topics—and the level of student turnover (mobility). The latter is important because more benefits 
accrued to students the longer they participated in Striving Readers. The cumulative benefits of longer 
exposure included increased familiarity with techniques and strategies, which allowed teachers to spend 
less time explaining the methods and more time carrying them out. In addition, of course, if the 
program does impact literacy skills, longer exposure would allow greater accrual of those benefits. 


School Characteristics 


One of the school characteristics most commonly cited by teachers and staff as influencing program 
implementation was size, which affected the proportion of literacy teachers and students the LIT was 
able to work with, as well as the composition of the grade-level and literacy teams. Another was teacher 
turnover rate. Schools with large numbers of new teachers (or teachers who were recently transferred 
into a Striving Readers grade level) were forced to continuously provide introductory trainings, which led 
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to frustration among experienced teachers looking for professional development at a higher level. (At 
two of the case study schools, 40% or more of their Striving Readers teachers were new to the school or 
to the program that year, and at all schools but one, at least one in four teachers were new to the 
program.) More broadly, teachers across case study schools were found to hold certain 
misunderstandings about the model (such as the role of the LIT), which may, in part, have been due to a 
lack of familiarity with the program. The percentage of newly certified teachers compared to veteran 
teachers at each school was also cited as being an important characteristic, particularly as a mediator of 
technology integration and use of differentiated instruction. Despite less general experience, newer 
teachers were described as being more open to and familiar with technology as a teaching tool, as well as 
other innovative techniques associated with the Striving Readers program. 


Concurrent Mandates, Programs, and Grants 


In some case study schools, the adoption of other literacy grants—even those focusing on different 
erades—were described as having a positive impact on Striving Readers. Reading First, for example, was 
described as placing a similar emphasis on differentiated instruction, thereby improving the school’s 
overall attentiveness to this method. The district’s newly established policy to departmentalize middle 
school grades® also had a substantial impact on some of the case study schools, particularly those that 
had not been departmentalized previously. In some instances, departmentalization fostered the spread of 
Striving Readers techniques and strategies to the content areas, as teachers trained in the model began to 
teach science, math, and social studies in addition to ELA, and were able to transfer the literacy practices 
they had used in ELA to these other content areas. This policy also had a negative effect to the extent 
that it reduced the amount of uninterrupted class time dedicated specifically to literacy instruction during 
ELA classes. 


Measures of Literacy Gain Among Case Study Schools. Table 24 presents the mean 
difference in ISAT reading scores, by grade, for the 2009-2010 academic year cohorts (immediately prior 
to the start of Striving Readers) compared with the 2005—2006 academic year cohorts. 


6° Departmentalization was mandated by the Chicago Public School system for all middle school grades beginning in the 
2009-2010 school year and entailed restructuring classrooms so that each subject is taught independently by dedicated 
teachers during single classroom blocks. 
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Table 24: Mean Differences in ISAT Reading Scale Scores: 2009-10 Cohorts 
Compared to 2005-06 Cohorts 


Case Study Schools All All Other 
All Non-Case All Non-SR 
Study Control Program 
Case Study figura Schools District 
Schools Schools Schools 
6th 4.9 -15 5.5 6.8 -0.3 8.6 3.4 5.4 4.26 6.2 
7th -2.4 12.3 0.9 -6.0 1.2 -4.3 -0.3 -15 3.25 2.8 
8th 2.3 4.9 -2.35 6.4 8.3 5.6 4.5 7 1.03 1.8 


At the eighth grade, where the potential duration of exposure to Striving Readers was greatest, the 
findings are highly positive: at five out of the six case study schools, the ISAT scores of their 2009-2010 
cohort of eighth graders exceeded those of their 2005-2006 (pre-program) cohort, and the magnitude of 
the mean difference between these cohorts was greater at five of the six case study schools than it was 
for the remaining treatment schools combined, which in turn exceeded the magnitude of the mean 
difference between eighth-grade cohorts at the control schools. Furthermore, at each grade there are 
particular schools where the mean difference between cohorts far exceeds that of the treatment group, 
control group, and district-wide means. Indeed, the overall mean difference between 2006 and 2010 
cohorts in the eighth grade for all case study schools combined (4.5 scale score points) was higher than 
the difference among non—case study treatment schools (1.7 points), higher than the control schools (1.0 
points), and higher than the remainder of the Chicago Public Schools district (1.8 points). These findings 
help substantiate the usefulness of looking to the case study schools for best-practice examples of how 
the Striving Readers model is best facilitated in different contexts. However, there are many reasons why 
these results cannot be assumed to be attributable to program impact. The most notable reasons are that 
the mean differences do not represent individual students’ gains, but compare cohorts of different 
students to each other; and that baseline equivalence has not been established between the control group 
and this subsample of treatment schools. Nevertheless, the findings provide preliminary evidence that 
Chicago Striving Readers might have positive, verifiable impact on student reading scores when it is 
implemented well. This hypothesis will be studied more rigorously during the final program year. 


Implications for Impact Analyses 


Results of the Year 4 implementation evaluation for Chicago Striving Readers revealed a number of 
important areas where there was evidence that program fidelity has been continuing to improve. 
Following refinements in fidelity scale definitions during Year 3, scores increased in several areas, 
sometimes dramatically, implying that some of the scores from Year 2 may have been under-estimated. 
Even in Year 4 (when scale definitions were changed only slightly), modest but fairly consistent increases 
were once again observed. Given that many of the staff had one more year of training and experience 
under their belts, it is reasonable to expect that their self-ratings of fidelity might be more accurate. 
Since increased knowledge is often known to have the paradoxical effect of /owering self-assessments as 
participants become more aware of what they do not know, the fact that fidelity scores still increased, 
even modestly, in Year 4 is all the more encouraging. To the extent that these improved scores are 
indeed a true reflection of program fidelity, it would be expected that the program should have a greater 
chance of achieving measurable impacts. 
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Nevertheless, there simultaneously remained a number of contravening factors that would have 
reduced the probability of achieving measurable impacts program-wide. The district continues to 
struggle with persistently high turnover rates among teachers. Among all Striving Reader schools, teacher 
turnover averaged 55% in the single year from Year 3 to Year 4 alone. While Year 4 was one of the most 
stable in terms of turnover among administrative staff, there were several cases where the principal or 
the LIT had changed, and continuity had already been lost as a result of turnover in these positions, as 
well as among District Coordinators, in previous years. This situation handicaps the program in a 
number of ways. The result is that staff at many schools have less experience with, and likely less 
understanding of the program. They would therefore tend to be less capable of providing accurate self- 
assessments and less likely to implement the program with fidelity, and the program would be less likely 
to become institutionalized. 


Staff turnover is quite likely one of the major reasons that substantial variations in fidelity among 
schools and program sub-components remained into Year 4. As a result of this variation, if the program 
model were effective, it would be reasonable to expect that the greatest impacts would occur at those 
schools with the highest fidelity; but when averaged with the lesser impacts of lower fidelity schools, they 
may become undetectable. 


Another important consideration for detecting impacts through a randomized controlled trial (RCT) 
is the contrast between the treatment and control groups. As has been discussed previously, since RCT 
studies measure differential, not absolute, impact, real impacts resulting from the program under study 
may be undetectable if the control group achieves similar impacts through other means. Striving 
Readers and control schools have been pursuing similar goals, and control schools have even been 
adopting some similar techniques, for several years, while all schools have been moving towards more 
rigorous instructional programs under the guidance of the Area Officers. During Year 4, control school 
principals have cited an increasing use of data-driven instruction, and some also cited the importance of 
literacy coaches who manage testing, help with professional development and facilitate staff 
collaboration. 


Nevertheless, there were also notable differences between Striving Readers and control school 
principals’ descriptions of their literacy programs. While many control schools employ literacy coaches, 
these individuals are not typically serving as interventionists. Some of the needs that control school 
principals cited for their literacy programs also serve to highlight some of the benefits provided by 
Striving Readers, at least in theory. These included a need for literacy specialists dedicated to the middle 
grades, staff training on differentiation, greater technology integration, more purposeful use of student 
data, and more print resources to support the integration of literacy in the content areas. 


Finally, there were important differences in the strategies used in supplemental literacy programs 
provided by Striving Readers and control schools. While, like Striving Readers, most control schools 
also reported providing after-school (or before-school) literacy programs, the eligibility criteria and focus 
of these programs were often quite different. Among control schools, rather than targeting them 
specifically for struggling readers, after-school programs most commonly were grouped heterogeneously, 
allowing open enrollment regardless of literacy level; or in at least one case, specifically including 
stronger readers as a way of conveying high expectations. Other programs focused on students who were 
just below grade level or at the threshold between performance levels, rather than being targeted to all 
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students who struggle with reading. It may be telling that control school principals’ confidence in the 
effectiveness of their schools’ overall literacy programs for struggling readers was often low. 


Given the limited data that could be obtained from control schools in this study, it is difficult to 
know where the balance falls in the similarities and differences between Striving Readers and control 
schools. Some of the evidence discussed here seems to indicate that it is exactly in the realm of serving 
struggling readers—the primary purpose of the Striving Readers program—where the differences may be 
the most substantial. These differences would seem to provide the contrast needed to assess Striving 
Readers’ impact for such students. Unfortunately, the fact that the only available outcome measure that 
is administered to all students—the ISAT—is thought to be a poor assessment for struggling readers 
clearly makes the opportunity less useful. 
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IV. Evaluation of the Overall Program Impact 
of the Chicago Striving Readers Initiative in Year 4 


Study Design 


Research Questions 


The research questions relating to assessment of the overall impact of the Chicago Striving Readers 
initiative that were explored during the first four years of the evaluation are presented in Table 25, below. 
In Year 2, research questions relating to exploring the initiative’s differential impact on students in 
different demographic subgroups were added to the study. In Year 3, an additional research question 
regarding the program impact on students who had the opportunity to receive two years of the blended 
intervention was investigated. In Year 4, research questions were added to examine the program impacts 
on students who had the opportunity to receive the blended intervention for either one year or three 


years. 
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Table 25: Research Questions Relating to the Overall Program Impact 
of the Chicago Striving Readers Initiative 


Years 2 and 3 Year 4 


Year | 


1) What is the impact of the Chicago 
Striving Readers program on sixth- 
through eighth-grade students’ (i.e., 
school-wide impact) reading scores, as 
measured by ClassViews and the ISAT? 


1) What is the impact of the Chicago 
Striving Readers program on sixth- 
through eighth-grade students’ (i.e., 
school-wide impact) reading scores, as 
measured by the ISAT"? 


1) What is the impact of the Chicago 
Striving Readers program on sixth- 
through eighth-grade students’ (i.e., 
school-wide impact) reading scores, as 
measured by the ISAT"? 


2) Is there a differential impact of the 
Chicago Striving Readers program on 
sixth- vs. seventh- vs. eighth-grade 
students’ reading scores (i.e., grade 
impact), as measured by ClassViews 
and ISAT? 


2) Is there a differential impact of the 
Chicago Striving Readers program on 
sixth- vs. seventh- vs. eighth-grade 
students’ reading scores, as measured 
by the ISAT#!? 


2) Is there a differential impact of the 
Chicago Striving Readers program on 
sixth- vs. seventh- vs. eighth-grade 
students’ reading scores, as measured 
by the ISAT#?? 


3) Is there a differential impact of the 
Chicago Striving Readers program on 
the reading scores of sixth- through 
eighth-grade students in different 
NCLB subgroups, including gender, 
race/ethnicity, socioeconomic status, 
special education status, and ELL 
status, as measured by the ISAT"? 


3) Is there a differential impact of the 
Chicago Striving Readers program on 
the reading scores of sixth- through 
eighth-grade students in different 
NCLB subgroups, including gender, 
race/ethnicity, socioeconomic status, 
special education status, and ELL 
status, as measured by the ISAT"? 


4) What is the impact of the Chicago 
Striving Readers program on the 
reading scores of students who had 
the opportunity to participate for one 
year (entered grade 6 either in SY 
2008-2009 or in SY 2009-2010), as 
measured by the ISAT"? 


4) What is the impact of the Chicago 
Striving Readers program on the 
reading scores of students who had 
the opportunity to participate for two 
years (entered grade 6 in SY 2007- 
2008), as measured by the ISAT™!? 


5) What is the impact of the Chicago 
Striving Readers program on the 
reading scores of students who had 
the opportunity to participate for two 
years (entered grade 6 in SY 2008- 
2009), as measured by the ISAT™!? 


6) What is the impact of the Chicago 
Striving Readers program on the 
reading scores of students who had 
the opportunity to participate for 
three years (entered grade 6 in SY 
2007-2008), as measured by the 
ISAT&2 


[?] As discussed above, ClassViews has not been used as an outcome measure since Year 2 because it is not administered 


consistently outside of Striving Readers schools. 
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Summary of Analytic Approach to the Impact Analysis 


Hierarchical linear modeling (HLM) was used to assess the overall impact of the Striving Readers 
Initiative on students’ reading performance, as well as whether there were differential impacts of the 
program on different grade-level and No Child Left Behind (NCLB) subgroups. These impact analyses 
focused on the intent-to-treat ITT) populations that are described later in this section. Because school 
was the unit of assignment but impacts are measured at the student level, two-level models were used for 
these cross-sectional analyses in order to account for the clustering of students within schools. For all 
HLM analyses, listwise deletion was used to remove students with missing data from the analytic 
samples. 


Main effects models were fitted to address the research questions regarding the overall program 
impacts of the Chicago Striving Readers initiative. Separate analyses were conducted to assess the overall 
impacts on students who had the opportunity to participate in the program for either one year, two 
years, or three years, in addition to examining the average impact of the blended intervention on all 
students with different amounts of intended treatment at the end of the fourth year. Additional analyses 
exploring the interactions between treatment and subgroups were also conducted to address research 
questions pertaining to differential impact by grade and by demographic group. Results of these latter 
analyses are presented under “Additional Analyses” later in this section. 


A number of covariates were included in the HLM models to assess the overall program impact on 
students. At the student level, the full models included the following covariates: baseline reading score, 
baseline math score, grade level, types of interventions being offered, gender, race/ethnicity,” special 
education status,’! eligibility for free or reduced-price lunch, and English language learner status. At the 
school level, the full models included the following variables: proportion of minority (non-White) 
students, proportion of female students, proportion of students reading at or above grade level, 
proportion of special education students, proportion of ELLs, proportion of students eligible for 
free /reduced-price lunch, school size in targeted grades, school cohort, and treatment. The additional 
analyses examining the differential program impact on the subgroups of students also included terms for 
the interactions between treatment and each subgroup indicator. 


Appendix O describes in greater detail the approaches used to fit these models, the specification of 
the models, the selection of covariates, and the treatment of missing data. An overall description of the 
intent-to-treat samples used for these analyses is provided below. 


Sampling Plan 


To facilitate an unbiased assessment of the impacts of Striving Readers, the study randomly assigned 
participating schools to either a treatment or control condition. As described in the Year 1 Impact 
Report.’ 


7 American Indian, Asian, White, and Other/Multiracial were combined into one category due to small sample sizes. 
71 Identified as students with an Individualized Education Plan (IEP). 


” Learning Point Associates, December 2007. 
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The process of random assignment began with CPS generating a list of schools having a high 
percentage of struggling readers, as measured by their performance on a standardized test. These 
schools were invited to participate in the program and informed that by agreeing to participate, they also 
were agreeing to be either a treatment or a control school. Control schools were promised a stipend of 
$5,000 per year for their assistance with the evaluation. From this pool of eligible and willing 
participants, treatment and control schools were randomly selected by LPA. Willingness to participate in 
the activities of the program and the evaluation was confirmed in writing during an informational session 
run by the CPS Office of Literacy. 


As further detailed in Appendix A of the Year 1 report, a power analysis established that a minimum 
sample of 32 schools (16 treatment and 16 control) would achieve adequate power (approximately 85%) 
to detect an effect size of roughly one-third standard deviation change over time with alpha = 0.05. 
Random assignment of schools occurred over two years, creating two cohorts of schools. Thirty-two 
schools were randomly assigned to either the treatment or control condition in school year 2006—2007 
(Cohort 1), and another 32 schools were randomly assigned in 2007-2008 (Cohort 2). As previously 
discussed, however, one of the Cohort 2 schools assigned to the treatment group did not join the study, 
so the resulting sample included a total of 31 treatment (Striving Readers) and 32 control schools.” 


The intent-to-treat analyses of the overall program impacts include all students in grades 6-8 in the 
63 study schools who were eligible for services according to the tier assignment criteria in respective 
school years (described in greater detail in Section I, above). In Striving Readers schools, students at all 
three tiers were exposed to the whole-school intervention. Students at Tier 2 were offered the whole- 
school intervention with additional support through scaffolded instruction that is provided during the 
regular classroom reading block (the targeted intervention). The students in Tier 3, the lowest 
performing students, were offered the intensive intervention (the AMP after-school program) in addition 
to the targeted and whole-school interventions. Therefore, the program impacts estimated by these 
analyses are actually the overall impacts of the integrated model of blended interventions on students at 
all three tiers while controlling for the types of interventions being offered to individual students.’4 


Sample Size and Power 


Table 26 below shows the number of intent-to-treat students who were included in the various analytic 
samples used to answer research questions about the impact of the blended intervention on students 
across tiers. Data are also disaggregated by treatment group and cohort. 


7 Although the 32nd treatment school did not participate in the study, it dropped out after the random assignment 
process. Nevertheless, comparisons of students in the 31 treatment and 32 control schools revealed that the two 
groups remained comparable on all measured characteristics (see Table 24). In addition, several other schools have 
since dropped out of the study, because they either closed or became “turnaround” schools, including four control 
schools and two treatment schools (see Section II for details). However, since the intent-to-treat population is defined 
at the time of random assignment (at the beginning of the study), students who were in these schools before they 
closed are still included in the intent-to-treat analyses to the extent possible, and restructured schools are also still in the 
ITT database. 


™ Tn these intent-to-treat analyses, targeted and intensive interventions were two covariates in the student-level model. 
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Table 26: Numbers of Intent-To-Treat Students with Complete Data"! by 
Treatment Group and Cohort 


Treatment Schools Control Schools 
Population Group 


Total Cohort | | Cohort 2 | Total Cohort | Cohort 2 


All students at Tiers |-3 in 
grades 6-8 at the end of SY 4,074 2,535 1,539 4,053 2,457 1,596 
2009-2010 (Analytic Group |) 


All students entering grade 6 
either in SY 2008-2009 or SY 2,693 1,767 926 2,692 1,667 1,025 
2009-2010 (Analytic Group 2) 


All students entering grade 6 in 
SY 2008-2009 1,318 859 459 1,281 775 506 
(Analytic Group 3) 


All students entering grade 6 in 
SY 2007-2008 1,316 77\ 545 | ,287 759 528 
(Analytic Group 4) 


cy 


‘Baseline scores, tier assignments, demographics, and outcome data were available for all students in this analysis. 


As shown in Table 26, there were a total of 4,074 Striving Readers students and 4,053 control 
students in the Year 4 study with complete data (respectively representing 76% and 77% of the total 
intent-to-treat populations).’> This final analysis sample (Analytic Group 1) included 2,535 treatment 
and 2,457 control students in Cohort 1, and 1,539 treatment and 1,596 control students in Cohort 2. 
The reasons for the difference between the original intent-to-treat populations and the final analysis 
sample are summarized in the flowchart in Appendix P, page P-1. 


The following groups of students were included in this intent-to-treat analytic sample: (1) students who were assigned 
to a study school (i.e., treatment or control school) in school year 2006-2007 (for Cohort 1 schools) or school year 
2007-2008 (for Cohort 2 schools) and were in one of the target grades in school year 2009-2010; (2) students who were 
new to one of the study schools in school year 2007-2008 and were in one of the target grades in school year 2009- 
2010; (3) students who were new to one of the study schools in school year 2008-2009 and were in one of the target 
grades in school year 2009-2010; and (4) students who were new to one of the study schools in school year 2009-2010 
and were in one of the target grades in school year 2009-2010. Only students with complete data were included in the 
HLM analyses. 
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Also shown in Table 26, analyses were conducted for three additional analytic groups: students who 
had the opportunity to receive either one year (Analytic Group 2), two years (Analytic Group 3), or three 
years (Analytic Group 4) of the blended intervention.”© (Note that the Analytic Group 1 was used to 
address research questions 1-3, while Analytic Groups 2, 3, and 4 were used to answer research 
questions 4, 5, and 6, respectively.) The flowcharts in Appendix P, pages P-2 — P-4 summarize the 
reasons for the differences between the original intent-to-treat populations and these final analysis 
samples. 


In order to determine the probability of detecting real treatment effects in the tested populations, 
power analyses were carried out for each analysis model that was conducted. Power analyses were 
conducted using Optimal Design software for cluster randomized trials.’” All power analyses used the 
following assumptions: 


¢ = ‘Two-level HLM model (students within schools) 
e =6Type I error rate (alpha) = 0.05 
e =Intra-class correlation (rho) = 0.05 


e Number of clusters (schools) = 63 


The power to detect an effect size of at least .33, as well as the smallest effect size that could be detected 
at an acceptable power level of 80%, were determined. Results of the power analyses are presented 
below. 


Analytic Group |: All students at Tiers |-3 in grades 6-8 at the end of SY 2009-2010 


(average impact of the blended intervention over different doses at the end of the 
fourth project year) 


Additional assumption: Cluster size (”) = 129 (average number of students in grades 6-8 and in Tiers 1— 
3 per school at end of SY 2009-2010) 


Results: 
Effect size = .333 Power = 100% 
Effect size = .171 Power = 80% (acceptable power level) 


76 These three analytic groups consisted of subsamples of Analytic Group 1. Specifically, Analytic Group 2 included 
students who were new sixth graders assigned to one of the study schools either in school year 2008-2009 or in school 
year 2009-2010 and were in one of the targeted grades in school year 2009-2010 (two student cohorts combined); 
Analytic Group 3 included students who were new sixth graders assigned to one of the study schools in school year 
2008-2009 and were in one of the targeted grades in school year 2009-2010 (one student cohort); Analytic Group 4 
included students who were new sixth graders assigned to one of the study schools in school year 2007-2008 and were 
in one of the targeted grades in school year 2009-2010 (one student cohort). 


7 Retrieved from http://sitemaker.umich.edu/group-based/optimal design software. 
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Analytic Group 2: All students entering sixth grade either in SY 2008-2009 or in SY 


2009-2010 (l-year overall program impact) 


Additional assumption: Cluster size (”) = 85 (average number of students entering sixth grade per 
school either in SY 2008-2009 or in SY 2009-2010) 


Results: 
Effect size = .333 Power = 100% 
Effect size = .177 Power = 80% (acceptable power level) 


Analytic Group 3: All students entering sixth grade in SY 2008-2009 (2-year overall 


program impact) 


Additional assumption: Cluster size (”) = 41 (average number of students entering sixth grade per school 
in SY 2008-2009) 


Results: 
Effect size = .333 Power = 100% 
Effect size = .194 Power = 80% (acceptable power level) 


Analytic Group 4: All students entering sixth grade in SY 2007-2008 (3-year overall 


program impact) 


Additional assumption: Cluster size (”) = 41 (average number of students entering sixth grade per school 
in SY 2007-2008) 


Results: 
Effect size = .333 Power = 100% 
Effect size = .194 Power = 80% (acceptable power level) 


Power analysis results show that there is sufficient statistical power to detect a program effect of 
less than one-fifth standard deviation for all analytic groups. 


Description of the Counterfactual 


Parallel to the implementation of Striving Readers in treatment schools, control schools have also 
engaged in school-wide literacy initiatives and a number of cases have adopted literacy interventions that 
incorporate some of the key elements emphasized in the Striving Readers model. As discussed in 
Section III above, under an RCT evaluation, such similarities may result in making some of the 
ptogram’s impacts undetectable. Control schools, for example, reported increasing efforts to incorporate 
the use of student data to drive instruction into their school’s culture. Examples of other literacy 
initiatives that control schools are currently implementing include: Scholastic Reading Inventory, Read 
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180, Reading First, Lucy Hawkins Writing Series, the Balanced Literacy model, and Lexia, among others. 
A more detailed description of the similarities and differences between treatment and control schools is 
provided under the “implications for impact analyses” section. 


Data Collection Plan 


The main outcome measure for student impact is reading scores on the Illinois Standards Achievement 
Test (ISAT), a statewide exam administered to all students in Chicago in grades 6 through 8 in the spring 
of each year.’8 The 2010 ISAT in reading is comprised of items from the SAT-10, published by Pearson 
Education, and items written by Illinois teachers. Pearson Education has created a standard, norm- 
referenced, abbreviated version of the SAT-10 assessment (SAT-10/Abb), which comprises 30 items 
that are also consistent with Illinois Assessment Frameworks. The inclusion of these 30 SAT-10 items 
on the ISAT allows for the reporting of nationally norm-referenced results such as national percentile 
ranks (ISBE, 2008). The overall ISAT exam in reading is made up of multiple-choice items (51 items in 
each grade for grades 5 though 8) as well as one extended-response item for each grade. Scoring of the 
ISAT is based on the number of correct items, with weighting based on item difficulty. Because test 
items change each year, the Rasch model of item response theory is applied to the ISAT multiple-choice 
tests in order to equate the scores from year to year and thereby create a vertically scaled test that allows 
for longitudinal comparisons. Details of the test development can be found in the technical manual 


(ISBE, 2007). 


Students’ ISAT achievement in reading is measured using performance levels of exceeds standards, meets 
standards, below standards, and academic warning. AIL ISAT treading scores are also reported on a continuous 
vertical scale from grade 3 through grade 8. Although the minimum score has been set at 120 for all 
grades and subjects, the maximum score is determined based on each year’s ISAT data. Table 27 
presents the scale score ranges for reading performance for grades 3 through 8. 


78 Prior to school year 2007-2008, English language learners in Illinois took a different language test (the IMAGE) in 
place of the ISAT. As of spring 2008, however, all ELL students took the ISAT in addition to the ACCESS. 
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Table 27: Scale Score Ranges Defining Student Performance Levels on the 2010 
ISAT Scales 


3 120-155 156-190 191-226 227+ 
4 120-157 158-202 203-236 237+ 
5 120-160 161-214 215-246 247+ 
6 120-166 167-219 220-256 257+ 
7 120-173 174-225 226-266 267+ 
8 120-179 180-230 231-277 278+ 


“Table retrieved from http://www.isbe.state.il.us/assessment/pdfs/cut_points_1|0.pdf. 


Description of the Fourth-Year Samples 


Description of Study Schools 


Overall, characteristics of treatment and control schools were very similar in most respects, with 
the exception of ethnic distribution: treatment schools had somewhat larger proportions of 
Hispanic students and somewhat smaller proportions of African American students than control 
schools. 


Because the proportion of students in a school who possess certain characteristics can influence the 
school environment in ways that affect all students over and above the ways that these characteristics 
might affect the individual students who possess them, we examined the proportion of students in each 
school (within the grades targeted by Striving Readers) who were female, English language learners 
(ELL), special education, or low income, as well as racial and ethnic distributions and school-wide 
attendance rates. We also examined school size in the targeted grades because, among other reasons, 
total enrollment can be a significant factor for a Striving Readers program, as it determines the number 
of students for whom each school’s LIT is responsible. School characteristics by cohort and treatment 
group are presented in Table 28. These data are presented for individual schools in Appendix Q, Tables 


Q1-Q-4. 
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Table 28: School Characteristics: Summary by Treatment Group by Cohort 


Target Grades 6-8 for School Year 2009-2010 


% Race/Ethnicity 


Total % % % Special k%lLow % 
N Female | LEPIl Education | Income | Attendance | Amer. African Other/ 
Indian Asian American | Hispanic | White | Multiracial 
I 2969 | 51% 10% 14% 93% 94.8% 1% 1% 43% 48% 7% 0% 
Control 
2 1844 | 49% 2% 17% 98% 92.7% <1% <1% 93% 7% <1% 0% 
I 3218 | 48% 10% 14% 95% 95.7% 1% 2% 23% 70% 4% 0% 
Treatment 
2 1687 | 50% 1% 17% 98% 93.4% <1% <1% 98% 2% <1% 0% 
Control 4813 | 50% 6% 15% 95% 93.9 1% 1% 62% 32% 4% 0% 
Overall 
Treatment 4905 | 49% 7% 15% 96% 94.9 <1% 2% 48% 47% 3% 0% 
t-statistic -1.870 -0.219 -1.226 -0.278 -1.576 0.371 -0.479 | 0.567 -0.856 0.561 | -- 
Treatment 
vs. Control df 58 58 58 58 58 58 58 58 58 58 -- 
(overall) 
p-value 0.067 0.827 0.225 0.782 0.120 0.712 | 0.634 0.573 0.396 0.577 | -- 


& Limited English proficiency 
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Number and Basic Characteristics of Students in Fourth-Year Sample 


The Striving Readers and control students who were included in ITT analyses were statistically 
comparable on all tested demographic variables, as well as on baseline reading and math 
performance. 


The demographic characteristics of all Striving Readers and control students, across all three tiers 
and across both cohorts, are summarized in Table 29.7? As these data show, the overall Striving Readers 
and control groups were demographically very similar.°° 


When looking at differences by cohort, results show that Cohort 2 schools were selected from an 
applicant pool that represented African American students much more heavily and Hispanic students 
less heavily than the Cohort 1 schools. This was also associated with a larger proportion of ELL students 
in Cohort 1 schools. 


” Table 24 presents detailed descriptive data by cohort, while Table 24a reports the baseline equivalence test results. 
Both focus on Analytic Group 1. 

50 Differences between treatment and control schools (across cohorts) were not statistically significant (p < .05) on most 
of these demographic characteristics, based on independent samples #tests conducted on school-level data that were 
reported in the Year 3 study, demonstrating that the school-level random assignment yielded two very similar groups 
of schools. 
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Table 29: Characteristics of Intent-To-Treat Students with Complete Data:"! Tiers 
1-3 


All Students | Cohort | Students | Cohort 2 Students | 
Characteristics | Control arene Control Treatment | Control Treatment 
(N=4053) |(N=4074) |(N=2457) |(N=2535) |(N=1596) | (N=1539) 


Grade level 


American Indian 


Asiati 


African ‘American 


Race/Ethnicity 
Hist anic 


Ite 


pone cere ae ised noni inn nvm neta ts | ama comiemaan in Smininameameniaied nu ndu unin na 


Mean baseline 
reading scores 


Mean baseline 
math scores 


®] Baseline scores, tier assignments, and outcome data were available for all students, making it possible to include 
them in impact analyses. 


Baseline Equivalency Tests 


Additional baseline equivalency tests were conducted at the student level for each analytic sample; results 
are presented in Tables 30a through 30d. It was observed that, for all analytic samples, the treatment 
students and their control counterparts were balanced on baseline tests in both reading and math. 
Baseline equivalence was also well established for most student characteristics. Although the two groups 
showed statistically significant differences in some demographic characteristics (e.g., race/ethnicity and 
free/reduced lunch eligibility status), since these variables were included as student-level covariates in the 
HLM analyses, they should not bias the impact estimates. 
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Table 30a: Baseline Equivalence for Analytic Group |—ITT Students at Tiers |-3 


in Grades 6-8 in SY 2009-2010 


(average impact of the blended intervention over different doses at the end of 


1?-statistic 


the fourth project year) 


Characteristics | 


Race/Ethnicity™ 


Special education 


Baseline Tests"! 


Baseline reading — Grade 6 
Baseline reading — Grade 7 


Baseline reading — Grade 8 


Baseline mathematics — Grade 8 


Control 
(N=4053) 


Treatment 
(N=4074) 


Control Treatment 
mean score __|mean score 


218.51 
218.96 
214.90 


227.92 
228.54 
224.46 


ll American Indian, Asian, White, and Other/Multiracial were combined into one category when conducting the 
chi-square test. 


2.404 


174.635 


2.053 
1.509 
5.723 


2.567 


t-statistic 


1.256 


0.955 


-0.487 
0.458 


-0.410 
-0.275 


2687 
2855 
2579 | 
2687 
ness 
2579 


0.121 


0.000 


0.152 


0.277 


p-value 


0.219 
0.017 


0.209 
0.627 
0.647 
0.340 
lo6e2 
0.784 


'®] Students’ baseline test scores were based on the ISAT scores from the spring prior to being assigned to a study 


school. 
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Table 30b: Baseline Equivalence for Analytic Group 2—ITT Students Entering 
Grade 6 at Tiers |-3 either in SY 2008-2009 or SY 2009-2010 


(l-year overall program impact) 


Characteristics 


Gender 


Race/Ethnicity"! 


Other/Multiracial 


Special education 


English language learners 


Free/reduced-price lunch eligible 


Baseline Tests '”! 


Baseline reading 


Baseline mathematics 


fl American Indian, Asian, White, and Other/Multiracial were combined into one category when conducting the 


chi-square test. 


Treatment 
(N=2693) 


Control 
N=2692) 


Control Treatment 


mean score mean score 


?-statistic 


4.239 


149.836 


3.987 
1.709 


t-statistic 


0.531 
0.668 


5.067 


0.040 


0.000 


0.046 
0.191 


p-value 


0.596 


0.024 


0.504 


(l Students’ baseline test scores were based on the ISAT scores from the spring prior to being assigned to a study 


school. Therefore, the baseline scores for students entering grade 6 in SY 2008-2009 were based on the spring 


2008 examination, whereas the baseline scores for students entering grade 6 in SY 2009-2010 were based on the 


spring 2009 administration. 
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Table 30c: Baseline Equivalence for Analytic Group 3—ITT Students Entering 
Grade 6 at Tiers I-3 in SY 2008-2009 
(2-year overall program impact) 


ae Control Treatment 3 re 


Gender —_____ firmer 2.581 | 0.108 


Race/Ethnicity!) | ameanneenneeenennennnnnfne 1 ee 70.500 2 0.000 


Other/Multiracial 


Special education ; 4 10.932 | 0.001 

English language learners : ; 0.000 I 0.993 

Free/reduced-price lunch eligible : ; 4.979 I 0.026 

Baseline Tests "! oe [eas eeaiees df p-value 
mean score mean score 

Baseline reading -0.386 2597 0.700 

Baseline mathematics 0.178 2597 0.859 


ll American Indian, Asian, White, and Other/Multiracial were combined into one category when conducting the 


chi-square test. 
'l The baseline scores for students entering grade 6 in SY 2008-2009 were based on the spring 2008 ISAT 
examination. 
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Table 30d: Baseline Equivalence for Analytic Group 4—ITT Students Entering Grade 6 


at Tiers I-3 in SY 2007-2008 
(3-year overall program impact) 


Characteristics 


Gender 


Race/Ethn icity®! nnn 


Other/Multiracial 


Special education 


English language learners 


Free/reduced-price lunch eligible 


Baseline Tests '”! 


Baseline reading 


Baseline mathematics 


Control 
N=1287) 


Control 


mean score 


f'] American Indian, Asian, White, and Other/Multiracial were combined into one category when conducting the 


chi-square test. 


Treatment 
(N=1316) 


Treatment 
mean score 


0.045 | 
47.923 2 
1.892 | 
2.047 | 
0.049 | 
t-statistic  |df 
0.882 2601 
-0.209 2601 


0.833 


0.000 


0.169 
0.153 
0.825 


p-value 


0.378 


0.834 


l The baseline scores for students entering grade 6 in SY 2007-2008 were based on the spring 2007 ISAT 


examination. 


Impacts on Students 


There were no detectable overall impacts on reading performance for the group of all students 


(all tiers) who had been offered blended intervention services during school year 2009-2010; nor 
were there detectable impacts for groups of all sixth graders (all tiers) who had been offered 
services for either one, two, or three years. 
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In order to obtain unbiased estimates of the program impacts, all outcome analyses were based on 
the intent-to-treat (ITT) samples, using the four analytic groups defined above. Table 31 shows the 
estimates of the overall impacts of the Chicago Striving Readers program on each of these four analytic 


groups.®! 


51 Note that the outcome for the analyses of Groups 1, 3, and 4 were the ISAT scores in spring 2010. In the analysis of 
Group 2, for the students entering sixth grade at all tiers in school year 2008—2009, the outcome was their ISAT scores 
in spring 2009; while for the new sixth graders in school year 2009-2010, the outcome was the ISAT scores in spring 
2010. 


While Analytic Group 1 includes a small percentage of students who were retained in grade, Analytic Groups 2, 
3, and 4 did not include any retained sixth graders from the previous year; only new incoming sixth graders were 
included in the 1-year, 2-year, and 3-year impact analyses. 


For students who had been offered multiple years of intervention, i.e., Analytic Groups 3 and 4, the ITT 
samples were generated by identifying the students who were in the schools when they were randomized, and then 
following them for multiple years. For example, Analytic Group 4 identified new incoming sixth graders in school year 
2007-2008 who were tested in spring 2010 (whether or not they actually received the intended intervention or 
remained in their originally assigned schools). 


Due to changes in the tiering definition in the first three project years (see Figure 1), the ITT samples for 2-year 
and 3-year impact analyses did not combine the current cohort with any previous student cohorts. Therefore, the ITT 
sample for estimating the 2-year overall program impact (i.e., Analytic Group 3) was distinct from the analysis sample 
on which the 2-year impact was estimated as reported in the Year 3 study. 


Impact estimates are based on the final simple models that did not include interaction terms. These relevant 
full and final models are presented in Tables O-6 and O-7, and Table O-10 through Table O-15 in Appendix O. 
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Table 31: Estimated Overall Impacts of Striving Readers (Main Effects Models) 


ion- Correspondin 
Unadjusted Means Ree soe E : 


Population Group Adjusted Means NCE Scores fee 


Control Treatment Control ‘Treatment 


All students at Tiers 
1-3 in Grades 6-8 at 
end of SY 0910 


(Analytic Group I, 
average impact) 


-0.121 -0.006 (0.874 0.102 


237.64 (237.52  |45.962 


All students entering 
6" grade either in SY 
0809 or SY 0910 230.45 231.99 230.62 231.54 43.362 44.137 0.925 0.045 |0.363 |0.137 
(Analytic Group 2, 
one-year impact) 


All students entering 
6" grade in SY 0809 
(Analytic Group 3, 


237.40 235.37 [237.09 235.92 45.627 44.731_—‘|-1.174—-0.052 0.301 _—|o.138 
two-year impact) 


All students entering 
6" grade in SY 0708 
(Analytic Group 4, 


242.33 242.91 |242.61 242.88 44.047 44.307 [0.268 ~—fo.0l4_|o.718 —_|o.107 
three-year impact) 


ll Effect sizes included in this report were derived based on Glass’s A. They were calculated using point estimates of 
fixed effects divided by the standard deviation of the control group with complete data in each analytic sample. 

bl The power (MDES) in this table is the actual power the study has after implementation and data collection, measured 
by the minimum impact in standard deviation units that the study can detect with confidence. This calculation produces 
the MDES for a two-tailed test with 80% power, and with an alpha level of 0.05, and accounts for clustering and for the 
inclusion of the covariates in the model. 


As shown in Table 31, the analyses did not reveal any significant overall impact of the Striving 
Readers Initiative on students’ reading performance, as measured by their ISAT reading outcome scale 
scores, for any of the four analytic groups. The differences in performance between students in each 
treatment group and students in the respective control groups were not statistically significant and the 
effect sizes were very small. However, further examination of the covariates in the model provides a 
better understanding of the differential program impacts for various demographic subgroups. These 
results are discussed in the next section. 


Additional Analyses 


Among all students (all tiers) who had been offered blended intervention services during school 
year 2009-2010, sixth and eighth graders responded more positively to the program than those in 
seventh grade, and Hispanic students responded more positively than students who were 
neither Black nor Hispanic. However, the program could not be considered “effective” for 
grade 6 or grade 8 or Hispanic students, since impact was not significantly greater than for their 
control counterparts. 
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Additional analyses were conducted to assess whether there was a differential impact of the Striving 
Readers Initiative for students in different grades and in different NCLB subgroups. For these analyses, 
grade level and NCLB subgroup variables, among others, were entered into the models as covariates and 
their interactions with treatment were explored. Analyses were conducted for all students in the ITT 
sample at the end of the fourth project year (i.e., Analytic Group 1). Results are reported in Tables O-8 
and O-9 in Appendix O, which present results for the full model and for the final model, respectively. 


It can be seen that there was no overall treatment effect for all students in the ITT sample at the end 
of the fourth project year, which is consistent with the findings from the main effects model shown in 
Table 31. As shown in the final model (Table O-9 in Appendix O), however, the interaction between 
seventh grade and treatment was significantly negative and that between the Hispanic students and 
treatment was significantly positive, indicating that there were differential program impacts on different 
grades and different racial and ethnic groups. Specifically, a differential program impact was found for 
students who were in seventh grade in 2009-2010 compared with those in grades 6 and 8, and for 
Hispanic students compared with Black students and students who were neither Black nor Hispanic.*2 
However, it should be noted that these “differential impacts” might only reflect differences in 
performance when comparing seventh graders vs. other grades, or Hispanic students vs. other 
races/ethnicities (i.e., the difference between treatment and control students was greater for sixth and 
eighth graders than for seventh graders, and the difference was greater for Hispanic students than for 
Black students and students other than Black or Hispanic). It still would not demonstrate that the 
program was “effective” for sixth or eighth graders or for Hispanic students, because the average 
performance of the treated sixth or eighth graders was not statistically different from that of sixth or 
eighth graders in the control schools, and the average performance of the treated Hispanic students was 
not statistically different from that of Hispanic students in the control schools.83 


No other interactions between treatment and NCLB subgroups were statistically significant, implying 
that the program has not had a differential impact on any of these subgroups of students. 


82 Further investigation was conducted by changing reference groups in HLMs to compare between three grades and 
also between different racial and ethnic groups. 


83 Main treatment effects were tested for each subgroup (ie., for each grade and for each racial/ethnic group) using 
additional HLM analyses. None of these treatment effects were statistically significant. 
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V. Evaluation of the Impact 
of the Chicago Striving Readers Initiative on Struggling 
Readers in Year 4 


Study Design 


Research Questions 


The research questions relating to assessment of the impacts of the targeted and intensive 
intervention models that were explored during the first four years of the evaluation of the Chicago 
Striving Readers Initiative are presented in Table 32, below. In Year 2, research questions regarding the 
initiative’s differential impact on students in different demographic subgroups were added to the study, 
while additional research questions relating to the program impacts on students who had the opportunity 
to participate for two years were addressed in Year 3.84 In Year 4, additional research questions were 
added so that the study now investigates the program impacts on students who had the opportunity to 
receive different combinations of interventions for either one, two, or three years. 


54 Research questions relating to the program’s impact on classroom practices were discussed in Section HI. 
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Table 32: Research Questions Relating to Impact of the Targeted and Intensive Interventions! 


Year | 


| Years 2 and 3 


1) What is the combined impact of 
the whole-school and targeted 
interventions of the Chicago Striving 
Readers program on the reading 
scores of 6th-grade Tier 2 students 
who had the opportunity to participate 
for one year, as measured by the 
ISAT? 


Year 4 


1) What is the combined impact of 
the whole-school and targeted 
interventions of the Chicago Striving 
Readers program on the reading 
scores of 6th-grade Tier 2 students 
who had the opportunity to participate 
for one year, as measured by the 
ISAT? 


1) What is the combined impact of 
the whole-school, targeted, and 
intensive interventions of the Chicago 
Striving Readers program™! on the 
reading scores of 6th-grade Tier 3 
students who had the opportunity to 
participate for one year, as measured 
by ClassViews and the ISAT? 


2) What is the combined impact of 
the whole-school, targeted, and 
intensive interventions of the Chicago 
Striving Readers program on the 
reading scores of 6th-grade Tier 3 
students who had the opportunity to 
participate for one year, as measured 
by the ISAT? 


2) What is the combined impact of 
the whole-school, targeted, and 
intensive interventions of the Chicago 
Striving Readers program on the 
reading scores of 6th-grade Tier 3 
students who had the opportunity to 
participate for one year, as measured 
by the ISAT? 


3) Is there a differential combined 
impact of the whole-school, targeted, 
and intensive interventions of the 
Chicago Striving Readers program on 
the reading scores of 6th-grade Tier 3 
students, who had the opportunity to 
participate for one year, in different 
NCLB subgroups, including gender, 
race/ethnicity, socioeconomic status, 
special education status, and ELL 
status, as measured by the ISAT? 


3) Is there a differential combined 
impact of the whole-school, targeted, 
and intensive interventions of the 
Chicago Striving Readers program on 
the reading scores of 6th-grade Tier 3 
students, who had the opportunity to 
participate for one year, in different 
NCLB subgroups, including gender, 
race/ethnicity, socioeconomic status, 
special education status, and ELL 
status, as measured by the ISAT? 


4) What is the combined impact of 
the whole-school and targeted 
interventions of the Chicago Striving 
Readers program on the reading 
scores of 6th-grade Tier 2 students 
who had the opportunity to participate 
for two years, as measured by the 
ISAT? [1 


4) What is the combined impact of 
the whole-school and targeted 
interventions of the Chicago Striving 
Readers program on the reading 
scores of 6th-grade Tier 2 students 
who had the opportunity to participate 
for two years, as measured by the 
ISAT? [1 


5) What is the combined impact of 
the whole-school, targeted, and 
intensive interventions of the Chicago 
Striving Readers program on the 
reading scores of 6th-grade Tier 3 
students who had the opportunity to 
participate for two years, as measured 
by the ISAT? 


5) What is the combined impact of 
the whole-school, targeted, and 
intensive interventions of the Chicago 
Striving Readers program on the 
reading scores of 6th-grade Tier 3 
students who had the opportunity to 
participate for two years, as measured 
by the ISAT? 
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Year | 


| Years 2 and 3 


Year 4 


6) Is there a differential combined 
impact of the whole-school, targeted, 
and intensive interventions of the 
Chicago Striving Readers program on 
the reading scores of 6th-grade Tier 3 
students, who had the opportunity to 
participate for two years, in different 
NCLB subgroups, including gender, 
race/ethnicity, socioeconomic status, 
special education status, and ELL 
status, as measured by ISAT?! 


7) What is the combined impact of 
the whole-school and targeted 
interventions of the Chicago Striving 
Readers program on the reading 
scores of 6th-grade Tier 2 students 
who had the opportunity to participate 
for three years, as measured by the 
ISAT?! 


8) What is the combined impact of 
the whole-school, targeted, and 
intensive interventions of the Chicago 
Striving Readers program on the 
reading scores of 6th-grade Tier 3 
students who had the opportunity to 
participate for three years, as 
measured by the ISAT?!“ 


9) Is there a differential combined 
impact of the whole-school, targeted, 
and intensive interventions of the 
Chicago Striving Readers program on 
the reading scores of 6th-grade Tier 3 
students, who had the opportunity to 
participate for three years, in different 
NCLB subgroups, including gender, 
race/ethnicity, socioeconomic status, 
special education status, and ELL 
status, as measured by the ISAT? “4 


fl Because grant resources enable schools to consistently provide targeted and intensive intervention services 
only at grade 6, an assessment of the impact of these models at other grades is not supported at present. 

bl The original wording of this research question stated that they explored the “intensive intervention impact”; 
however, because Tier 3 students are also supported by the whole-school and targeted intervention models, 
the evaluation of the program impact on this particular group of students cannot isolate the impact of any 


particular intervention model. 


Il These include students who progressed to the seventh grade in their second year, as well as those who were 
retained in grade 6. For the same reason that the one-year analysis focused on grade 6, they did not include 
students who entered the program at grade 7 or 8. 

4] These include students who progressed to eighth grade in their third year, as well as those who were retained 
in grade 6 or 7. For the same reason that the one-year and two-year analyses focused on grade 6, they did not 


include students who entered the program at grade 7 or 8. 


162 


Summary of Analytic Approach to the Impact Analysis 


HLM analyses were again conducted to assess the combined impacts of the Striving Readers 
interventions on Tier 2 and Tier 3 students’ reading performance, as well as whether there were 
differential impacts of the interventions on different grade-level and NCLB subgroups. The impact 
analyses were based on the intent-to-treat samples that will be described later in this section. Because 
school was the unit of assignment but impacts are measured at the student level, two-level models were 
again used for these cross-sectional analyses in order to account for the clustering of students within 
schools. Listwise deletion was again used to remove students with missing data from the analytic 
samples. 


Main effects models were fitted to answer the research questions regarding the overall combined 
impacts of the whole-school, targeted, and intensive interventions on struggling readers who were either 
sixth-grade Tier 2 or sixth-grade Tier 3 students and had the opportunity to participate in the Chicago 
Striving Readers program for either one year, two years, or three years. Additional analyses exploring the 
interactions between treatment and subgroups were also conducted to investigate the differential impact 
on sixth-grade Tier 3 students by demographic group. Results of these latter analyses are presented 
under “Additional Analyses” later in this section. 


Similar to the HLM analyses conducted for the evaluation of the overall program impact in Section 
IV, the two-level models used to assess the program impact on sixth-grade Tier 2 and sixth-grade Tier 3 
students also included a number of pertinent covariates. At the student level, the full models included 
the following variables: baseline reading score, baseline math score, gender, race/ethnicity,®> special 
education status,®° eligibility for free or reduced-price lunch, and English language learner status. At the 
school level, the full models included the exactly same list of covariates as in the analyses of the overall 
program impact, in addition to the major treatment predictor. The further analyses examining the 
differential program impact on the subgroups of students also included terms for the interactions 
between treatment and each subgroup indicator. 


Appendix R describes in greater detail the approaches used to fit these models, the specification of 
the models, the selection of covariates, and the treatment of missing data. An overall description of the 
intent-to-treat samples used for these analyses is provided below. 


Sampling Plan 


The process for selection of participating schools and random assignment to treatment and control 
groups was described in Section IV, as were the definition of the intent-to-treat population and the 
process for matching to students at control schools. For the purpose of evaluating the program impacts 
on students eligible for the targeted and intensive interventions, only sixth-grade students at Tier 2 or 
Tier 3 were included in the corresponding analyses. 


85 American Indian, Asian, White, and Other/Multiracial were combined into one category due to small sample sizes. 


86 Tdentified as students with an Individualized Education Plan (IEP). 
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Because there is no group of students in the Chicago Striving Readers schools who receive either the 
targeted or intensive intervention exclusively (all students are also offered at least the whole-school 
intervention, and Tier 3 students have the opportunity to receive all three intervention models), it is not 
possible to isolate the impacts of any one intervention model. Rather, the analytic approach was 
designed to isolate the impacts of the Chicago Striving Readers initiative as implemented on specific 
groups of students. The analyses of impacts on Tier 2 and Tier 3 students focused on the sixth grade 
only, since LITs’ availability was less consistent in the higher grade classrooms in some schools. 
Specifically, the program focus expanded into grade 7 in Year 2 and to grade 8 in Year 3, but while most 
Cohort 2 classrooms were supported by the LIT, because of the larger size of the 16 Cohort 1 schools, 
seventh- and eighth-grade classrooms in most of these schools were not.8’ As a result, the targeted 
intervention was not implemented as consistently in grades 7 and 8, and for this reason, the impact 
analyses in this section did not include students who entered the program at grade 7 or 8. 


Sample Size and Power 


Table 33, below, presents the number of intent-to-treat students who were included in the various 
analytic samples used to answer research questions about the program impacts on sixth-grade Tier 2 and 
sixth-grade Tier 3 students. Data are also disaggregated by treatment group and cohort. 


87 Larger schools with more sections of sixth grade classes leave the LIT less available to provide support at Grades 7 
and 8. Nevertheless, all teachers are still able to participate in school literacy teams and in program-sponsored 
professional development. 
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Table 33: Numbers of Intent-To-Treat Sixth-Grade Tier 2 and Tier 3 Students with 
Complete Data! by Treatment Group and Cohort 


Treatment Schools Control Schools 
Population Group 


Total Cohort | | Cohort2 Total Cohort | | Cohort 2 


Students entering 6th grade at 
Tier 2 in SY 2008-2009 or SY | 660 412 248 628 376 252 
2009-2010 (Analytic Group 5) 


Students entering 6th grade at 
Tier 3 in SY 2008-2009 or SY | 602 336 266 632 318 314 
2009-2010 (Analytic Group 6) 


Students entering 6th grade at 
Tier 2 in SY 2008-2009 331 216 115 319 194 125 


(Analytic Group 7) 


Students entering 6th grade at 
Tier 3 in SY 2008-2009 298 157 141 284 134 150 


(Analytic Group 8) 


Students entering 6th grade at 
Tier 2 in SY 2007-2008 214 106 108 219 126 93 


(Analytic Group 9) 


Students entering 6th grade at 
Tier 3 in SY 2007-2008 452 208 244 452 214 238 
(Analytic Group 10) 


ll Baseline scores, tier assignments, demographics, and outcome data were available for all students in this 
analysis. 


As shown in Table 33, intent-to-treat analyses of impacts for struggling readers were carried out for a 
total of six analytic groups: students who had the opportunity to participate for either one year (Analytic 
Groups 5 and 6),88 two years (Analytic Groups 7 and 8),8° or three years (Analytic Groups 9 and 10).9° 
(Note that Analytic Groups 5, 7, and 9 were used to address research questions 1, 4, and 7, respectively, 


88 Analytic Group 5 included students who were new Tier 2 sixth graders assigned to one of the study schools either in 
school year 2008-2009 or in school year 2009-2010 and were in one of the targeted grades in school year 2009-2010 
(two student cohorts combined); Analytic Group 6 included students who were new Tier 3 sixth graders assigned to 
one of the study schools either in school year 2008-1009 or in school year 2009-2010 and were in one of the targeted 
grades in school year 2009-2010 (two student cohorts combined). 


8° Analytic Group 7 included students who were new Tier 2 sixth graders assigned to one of the study schools in school 
year 2008-2009 and were in one of the targeted grades in school year 2009-2010 (one student cohort); Analytic Group 
8 included students who were new Tier 3 sixth graders assigned to one of the study schools in school year 2008-2009 
and were in one of the targeted grades in school year 2009-2010 (one student cohort). 


°° Analytic Group 9 included students who were new Tier 2 sixth graders assigned to one of the study schools in school 
year 2007-2008 and were in one of the targeted grades in school year 2009-2010 (one student cohort); Analytic Group 
10 included students who were new Tier 3 sixth graders assigned to one of the study schools in school year 2007-2008 
and were in one of the targeted grades in school year 2009-2010 (one student cohort). Only students with complete 
data were included in the corresponding HLM analyses. 
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while Analytic Groups 6, 8, and 10 were used to answer research questions 2—3, 5—6, and 8-9, 
respectively.) All six of the above analytic groups consisted of subsamples of the analytic groups used to 
evaluate the overall program impact of one year, two years, or three years of the blended interventions in 
Section IV (i.e., Analytic Groups 2, 3, and 4, respectively). 


Power analyses for the outcome evaluation of the targeted and intensive interventions were 
conducted using the same assumptions as those used for assessing the overall program impact. 


Specifically: 


Two-level HLM model (students within schools) 
Type I error rate (alpha) = 0.05 

Intra-class correlation (rho) = 0.05 

Number of clusters (schools) = 63 


In each case, the power to detect an effect size of at least .33, and the minimum detectable effect size 
at an acceptable power level of 80%, were determined. Results of the power analyses are presented 


below. 


Analytic Group 5: Students entering 6th grade at Tier 2 either in SY 2008-2009 or in SY 2009- 


2010 (l-year whole-school plus targeted interventions impact) 


Additional assumption: Cluster size (7) = 20 (average number of students entering sixth grade at Tier 2 
per school either in SY 2008-2009 or in SY 2009-2010) 


Results: 
Effect size = .333 Power = 98% 
Effect size = .225 Power = 80% (acceptable power level) 


Analytic Group 6: Students entering sixth grade at Tier 3 either in SY 2008-2009 or in SY 2009- 


2010 (I-year whole-school plus targeted plus intensive interventions impact) 


Additional assumption: Cluster size (”) = 19 (average number of students entering sixth grade at Tier 3 
per school either in SY 2008-2009 or in SY 2009-2010) 


Results: 


Effect size = .333 Power = 98% 
Effect size = .227 Power = 80% (acceptable power level) 
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Analytic Group 7: Students entering sixth grade at Tier 2 in SY 2008-2009 (2-year whole-school 


plus targeted interventions impact) 


Additional assumption: Cluster size (”) = 10 (average number of students entering sixth grade at Tier 2 
per school in SY 2008-2009) 


Results: 
Effect size = .333. Power = 93% 
Effect size = .275 Power = 80% (acceptable power level) 


Analytic Group 8: Students entering sixth grade at Tier 3 in SY 2008-2009 (2-year whole-school 


plus targeted plus intensive interventions impact) 


Additional assumption: Cluster size (”) = 9 (average number of students entering sixth grade at Tier 3 


per school in SY 2008-2009) 


Results: 
Effect size = .333 Power = 91% 
Effect size = .285 Power = 80% (acceptable power level) 


Analytic Group 9: Students entering sixth grade at Tier 2 in SY 2007-2008 (3-year whole-school 


plus targeted interventions impact) 


Additional assumption: Cluster size (”) = 7 (average number of students entering sixth grade at Tier 2 


per school in SY 2007—2008) 


Results: 
Effect size = .333 Power = 86% 
Effect size = .309 Power = 80% (acceptable power level) 


Analytic Group 10: Students entering sixth grade at Tier 3 in SY 2007-2008 (3-year whole- 


school plus targeted plus intensive interventions impact) 


Additional assumption: Cluster size (”) = 14 (average number of students entering sixth grade at Tier 3 


per school in SY 2007—2008) 


Results: 
Effect size = .333 Power = 97% 
Effect size = .247 Power = 80% (acceptable power level) 


As these results show, the distribution of students and schools in this study provides sufficient 
statistical power to detect a program effect of approximately one third standard deviation and even 


smaller effective sizes could be detected for some analytic groups. 
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Description of the Counterfactual 


As described under the whole school blended intervention section, control schools have been adopting 
literacy interventions that often incorporate many of the key elements of the Striving Readers model, 
including the use of data to inform instruction, research-based literacy strategies and practices, 
technology integration, and professional development around these topic areas. These schools have also 
been implementing additional interventions for their struggling readers, which bear some similarities to 
the Striving Readers’ targeted and intensive interventions, but are also different in important ways. For 
example, some control school principals mentioned the use of reading and literacy coaches as an 
important component for the success of their school’s literacy efforts. The role of these coaches 
included leading professional development efforts, facilitating collaborative work among teachers, 
managing the testing for students and maintaining a resource room. However, although some of these 
responsibilities are also often in the purview of literacy intervention teachers, none of the control schools 
described their literacy coaches as providing in-class support for struggling readers or acting as 
interventionists. 


In addition, slightly over three-quarters of the control schools reported offering on-site literacy 
programs that provided increased instructional time before or after school. However, only one-fourth of 
these programs were targeting struggling readers exclusively. In many cases, there was either an open 
enrollment policy, or the eligibility criteria included broader considerations such as demographic 
requirements (e.g., free/reduced-lunch eligibility), space limitations, teacher referrals /recommendations, 
and student interest. All of these criteria tended to create more heterogeneous groupings than are found 
in the AMP classes. An additional discussion of the similarities and differences between treatment and 
control schools is provided under the “implications for impact analyses” section. 


Data Collection Plan 


The same reading assessment instrument, the ISAT, and the same data collection schedule were used for 
analyses of program impact on achievement of sixth-grade Tier 2 or Tier 3 students as previously 
described for the overall (blended) program impact analyses in Section IV. 


Description of Fourth-Year Samples 


Numbers and Basic Characteristics of Students 


School characteristics were presented in Section IV. Table 34, below, presents the demographic 
characteristics and baseline achievement data of Striving Readers and control students at Tiers 2 and 3 
(struggling readers) in grades 6 through 8 with complete data at the end of the fourth project year, 
disaggregated by treatment group and cohort. As these data show, the treatment and control students 
were demographically very similar, although there were a few notable differences. For example, a slightly 
larger percentage of students in control schools than treatment schools were African American, while a 
larger percentage of students in treatment schools than control schools were Hispanic. Data by cohort 
shows that almost all Tier 2 and 3 students in the targeted grades in Cohort 2 schools were African 
American, whereas Cohort 1 schools had similar percentages of African American and Hispanic students 
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at Tiers 2 and 3 in the targeted grades. In addition, consistent with their higher proportions of Hispanic 
students, Cohort 1 schools also had higher proportions of ELL students than Cohort 2 schools. 


Table 34: Characteristics of Intent-To-Treat Tier 2 and Tier 3 Students with 
Complete Data! 


All Students Cohort | Students Cohort 2 Students 


Characteristics Control Treatment |Control Treatment |Control Treatment 
(N= tis (N=1,754) |(N= gu (N=953)  |(N=859) _‘|(N=801) 


Female 
American Indian 


African American 
Race/Ethnicity 


Hispanic 


Other/Multiracial 


Special education 15.4% 17.6% 17.9% 18.0% 12.6% 17.0% 
Free/reduced-price lunch eligible 98.1% 98.4% 97.3% 97.6% 98.8% 99.4% 
200.41 200.65 201.01 201.42 199.65 199.55 
lean Paceline 203.22 204.00 202.71 204.69 203.73 203.12 
reading score 
Grade 8 201.18 201.80 199.08 200.03 203.27 203.33 
Grade 6 213.50 213.04 215.68 214.63 210.70 210.74 
Mean baseline 

Grade 7 214.29 214.01 215.31 215.47 213.26 212.16 
math score 
Grade 8 212.77 QZ 209.43 209.86 216.08 213.33 


ll Baseline scores, tier assignments, demographics, and outcome data were available for all students in this analysis. 


Baseline Equivalency Tests 


Tables 35a through 35f report the results from baseline equivalency tests for each of the six analytic 
samples. It was observed that, for each analytic sample, the treatment students and their control 
counterparts were balanced on baseline tests in both reading and math. Baseline equivalence was also 
well established for most student characteristics. Even where the two groups showed statistically 
significant differences in some demographic characteristics (e.g., race/ethnicity and special education 
status), the magnitude of the differences were generally not large; the differences should not bias the 
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impact estimates because these demographic variables were included as student-level covariates in all 
HLM analyses. 


Table 35a: Baseline Equivalence for Analytic Group 5—ITT Students Entering Grade 
6 at Tier 2 either in SY 2008-2009 or in SY 2009-2010 
(l-year whole-school plus targeted interventions impact) 


Characteristics pone! [eeaners 2_statisti -value 
(N=628) (N=660) —‘|* “Sttistic P 


Female : 
Gals 0.003 I 0.957 


Male 

American Indian 

Asian  —=—-*(0.00%——~™S : 
Race/Ethnicity"! ee sna 12.751 2 0.002 

Hispanic 40.15% 

White 2.58% 

Other/Multiracial [1.43% 
Special education 1.911 l 0.167 
English language learners 0.038 | 0.846 
Free/reduced-price lunch eligible 2.262 | 0.133 
Baseline Tests as reaement t-statistic |df p-value 

mean score |Mmean score 

Baseline reading -0.66 1286 0.509 
Baseline mathematics -0.185 1286 0.853 


ll American Indian, Asian, White, and Other/Multiracial were combined into one category when conducting 
the chi-square test. 
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Table 35b: Baseline Equivalence for Analytic Group 6—ITT Students Entering 
Grade 6 at Tier 3 either in SY 2008-2009 or in SY 2009-2010 
(I-year whole-school plus targeted plus intensive interventions impact) 


Characteristics pone reatmient 2-statistic |df value 
(N=632) (Ney) pisces Pe 
(Genel ere 3.382 | 0.066 
= rn yea 
Race/Ethnicity"! saa TN ee eee ees eT PW 0] Pd 2 0.000 
26.90% 
2.22% 
“TY? 
Special education 3.041 I 0.081 
English language learners 3.487 I 0.062 
Free/reduced-price lunch eligible 0.403 | 0.526 
Control 
Baseline Tests eae Hpcaanent t-statistic df”! p-value 
mean score mean score 
Baseline reading -0.677 1232 0.499 
Baseline mathematics 0.421 1227.33 |0.674 


ll American Indian, Asian, White, and Other/Multiracial were combined into one category when conducting 


the chi-square test. 


Pl Df were adjusted when Levene’s test of homogeneity of variances indicated unequal variances. 
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Table 35c: Baseline Equivalence for Analytic Group 7—ITT Students Entering Grade 


6 at Tier 2 in SY 2008-2009 


(2-year whole-school plus targeted interventions impact) 


Characteristics 


Female 


Male 


Gender 


American Indian 
Asian 


African American 


Race/Ethnicity™ 


Hispanic 
White 
Other/Multiracial 
Special education 
English language learners 


Free/reduced-price lunch eligible 
Baseline Tests 


Baseline reading 


Baseline mathematics 


Control 
(N=319) 


Control 


mean score 


Treatment 
(NEeEED) 


Treatment 


mean score 


x?-statistic 


0.053 


8.412 


5.489 


2.54 


t-statistic 


0.006 


-0.541 


0.702 648 


0.019 
0.937 


0.589 


0.818 


0.015 


0.111 


p-value 


0.483 


ll American Indian, Asian, White, and Other/Multiracial were combined into one category when conducting 


the chi-square test. 
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Table 35d: Baseline Equivalence for Analytic Group 8—ITT Students Entering Grade 


6 at Tier 3 in SY 2008-2009 


(2-year whole-school plus targeted plus intensive interventions impact) 


Characteristics 


Gender sisi ne 


Race/Ethnicity"! 


Other/Multiracial 
Special education 
English language learners 


Free/reduced-price lunch eligible 
Baseline Tests 


Baseline reading 


Baseline mathematics 


Control 
(N=284) 


Control 


mean score 


Treatment 


2. ain ae 
(N=298) X‘-statistic 


2.873 


=f 14.96 | 


2.052 


Treatment 
mean score 


t-statistic 


0.530 
1.550 


0.089 
0.167 


580 


PENNE 


0.090 


0.001 


0.152 


0.766 
0.683"! 


p-value 


0.596 


0.122 


ll American Indian, Asian, White, and Other/Multiracial were combined into one category when conducting 


the chi-square test. 


Pl Chi-square test result may be invalid because one cell has expected cell counts less than 5. 
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Table 35e: Baseline Equivalence for Analytic Group 9—ITT Students Entering 
Grade 6 at Tier 2 in SY 2007-2008 


(3-year whole-school plus targeted interventions impact) 


aa Control Treatment 2 a 
Conc 1 tee ean 2.495 0.114 
weno I a 2.679 2 0.262 
ee so 

Special education 0.082 I 0.774 

English language learners NA®l NA®)  |NA®! 

Free/reduced-price lunch eligible 1.500 I 0.221 
Control 

Baseline Tests Peeeomene t-statistic  |df* p-value 
mean score mean score 

Baseline reading 1.811 427.001 |0.071 

Baseline mathematics -1.558 431 0.120 


fl American Indian, Asian, White, and Other/Multiracial were combined into one category when conducting 
the chi-square test. 

bl Chi-square test was not conducted because all the students were non-ELLs. 

I] Df were adjusted when Levene’s test of homogeneity of variances indicated unequal variances. 
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Table 35f: Baseline Equivalence for Analytic Group 10—ITT Students Entering 
Grade 6 at Tier 3 in SY 2007-2008 
(3-year whole-school plus targeted plus intensive interventions impact) 


ae Treatment 2 ae 

Characteristics (N=452) x’-statistic [df p-value 

CC, AT erne as 1.787 | 0.181 

Race/Ethnicity!) | |e ne Le be 711 2 0.175 

ee 1. 

Special education 4.819 | 0.028 

English language learners 1.351 | 0.245 

Free/reduced-price lunch eligible 0.484 | 0.487 
Control 

Baseline Tests Tpesenent t-statistic {df p-value 
mean score mean score 

Baseline reading 0.073 902 0.942 

Baseline mathematics -1.291 902 0.197 


ll American Indian, Asian, White, and Other/Multiracial were combined into one category when conducting 
the chi-square test. 
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Impacts on Students 


There was a statistically significant overall impact on reading performance for the 
group of sixth-grade Tier 2 students who had been offered one year of the whole- 
school and targeted intervention services. However, no detectable overall impact was 
found for the group of sixth-grade Tier 3 students who had been offered one year of 
the whole-school, targeted, and intensive interventions, in part because many of them 
did not actually enroll in the after-school program. There were also no detectable 
impacts for groups of sixth graders (Tier 2 or Tier 3) who had been offered relevant 
services for either two years or three years. 


All outcome analyses in this section were also based on the intent-to-treat (ITT) samples in order to 
produce unbiased estimates of the program impacts. A total of six groups were analyzed in the impact 
evaluation of the targeted and intensive interventions, as shown in Table 36 below.?! 


°! Note that the outcome measure for the analyses of Groups 7 through 10 was the spring 2010 ISAT. In the analysis of 
Groups 5 and 6, for the students entering 6th grade at Tier 2 or Tier 3 in school year 2008-2009, the outcome 
measure was the spring 2009 ISAT; while for the new Oth graders at Tier 2 or Tier 3 in school year 2009-2010, the 
outcome was the spring 2010 ISAT. 


Analytic Groups 5 through 10 did not include any retained 6th graders from the previous year—only new 
incoming 6th graders were included in the 1-year, 2-year, and 3-year impact analyses. 


For students who had been offered multiple years of intervention, i.e., Analytic Groups 7 through 10, the ITT 
samples were generated by identifying the students who were in the schools when they were randomized, and then 
following them for multiple years. For example, Analytic Group 10 identified new incoming 6th graders at Tier 3 in 
school year 2007-2008 who were tested in spring 2010 (whether or not they actually received the intended intervention 
or remained in their originally assigned schools). 


Due to changes in the tiering definition in the first three project years (see Figure 1), the ITT samples for 2- 
year and 3-year impact analyses did not combine the current cohort with any previous student cohorts. Therefore, the 
ITT samples for estimating the 2-year combined impacts (i.e., Analytic Groups 7 and 8) were distinct from the analysis 
samples on which the corresponding two-year impacts were estimated and reported in the Year 3 study. 
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Table 36: Analytic Groups for Analyses of Impacts on Tier 2 and 3 Students 


Analytic Group 


Students in the Analyses 


Students entering 6th grade at 
Tier 2 either in SY 2008-2009 
or in SY 2009-2010 


Focus of the Analyses 


The analysis of this group investigates the |-year 
combined impact of the whole-school and targeted 
interventions. 


Students entering 6th grade at 
Tier 3 either in SY 2008-2009 
or in SY 2009-2010 


The analysis of this group explores the |-year combined 
impact of the whole-school, targeted, and intensive 
interventions. 


Students entering 6th grade at 
Tier 2 in SY 2008-2009 


The analysis of this group examines the 2-year combined 
impact of the whole-school and targeted interventions. 


Students entering 6th grade at 
Tier 3 in SY 2008-2009 


The analysis of this group explores the 2-year combined 
impact of the whole-school, targeted, and intensive 
interventions. 


Students entering 6th grade at 
Tier 2 in SY 2007-2008 


The analysis of this group investigates the 3-year 
combined impact of the whole-school and targeted 
interventions. 


Students entering 6th grade at 
Tier 3 in SY 2007-2008 


The analysis of this group examines the 3-year combined 
impact of the whole-school, targeted, and intensive 
interventions. 


Table 37 presents the estimates of the combined impact of the Chicago Striving Readers program on 
each of the six ITT analytic samples described above.% 


°2 Impact estimates are based on the final simple models that did not include interaction terms. These relevant full and 
final main effects models are presented in Table R-6 though Table R-9, Table R-12 though Table R-15, and Table R-18 


through Table R-21 in Appendix R. 
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Table 37: Estimated Impacts of Striving Readers on Tier 2 and Tier 3 Students 
(Main Effects Models) 


Regression- Corresponding 


Unadjusted Means : 
Population Group Adjusted Means 


Control ‘Treatment |Control ‘Treatment Control ‘Treatment 


NCE Scores Estimated Effect 


: -value 
Impact _ Size! P 


Students entering 6th 
grade at Tier 2 in SY 
2008-09 or in SY 
2009-10 (Analytic 
Group 5, one-year 
impact on Tier 2) 


2.156 0.174 |0.048 0.242 


222.49 (224.65 


Students entering 6th 
grade at Tier 3 in SY 
2008-09 or SY 2009- 
10 (Analytic Group 6, 
one-year impact on 
Tier 3) 


209.77 [210.10 |209.38 (21052 [25.469 (26.430 (1.139 ~—-|o.073._|o.295_Jo.i92 


Students entering 6th 
grade at Tier 2 in SY 
2008-09 (Analytic 229.61 228.36 229.85 228.55 40.083 39.087 -1.297 -0.089 |0.378 |0.281 
Group 7, two-year 
impact on Tier 2) 


Students entering 6th 
grade at Tier 3 in SY : : 
2008-09 (Analytic 215.28 214.37 216.18 214.26 29.614 28.144 -1.920 -0.113 0.224 {0.257 
Group 8, two-year ; 
impact on Tier 3) 


Students entering 6th 
grade at Tier 2 in SY 
2007-08 (Analytic 240.80 239.97 241.05 240.28 42.547 41.807 -0.77| -0.064 |0.544 0.295 
Group 9, three-year 
impact on Tier 2) 


Students entering 6th 
grade at Tier 3 in SY 
2007-08 (Analytic 226.40 226.60 226.69 226.73 28.738 28.777 0.044 0.003 |0.966 0.173 
Group 10, three-year 
impact on Tier 3) 


ll Effect sizes included in this report were derived based on Glass’s A. They were calculated using point estimates of 
fixed effects divided by the standard deviation of the control group with complete data in each analytic sample. 

bl The power (MDES) in this table is the actual power the study has after implementation and data collection, measured 
by the minimum impact in standard deviation units that the study can detect with confidence. This calculation produces 
the MDES for a two-tailed test with 80% power, and with an alpha level of 0.05, and accounts for clustering and for the 
inclusion of the covariates in the model. 


As shown in Table 37, results indicate that sixth-grade Tier 2 students in Striving Readers schools 
who had been offered one year of the whole-school and targeted interventions significantly 
outperformed their counterparts in control schools, although the ITT analyses did not reveal any 
significant overall impact of the program on the reading achievement for the other analytic groups (one- 
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year impact for Tier 3 students, or two- and three-year impacts for Tier 2 and 3 students). It should be 
noted, however, that average impacts for Tier 3 students would have been lower because of their low 
participation rates in AMP. Further examination of the covariates in the model may provide a better 
understanding of the differential program impacts on various demographic subgroups. These results are 
discussed in the following section. 


Additional Analyses 


Among sixth-grade Tier 3 students who had been offered one year of the whole-school, targeted, 
and intensive intervention services, students who were not eligible for free or reduced-price lunch 
responded more positively to the program than those who were eligible. Among sixth-grade Tier 3 
students who had the opportunity to participate for three years, female students responded more 
positively to the program than male students. However, the program could not be considered 
“effective” for students in either of these subgroups, because impact was not significantly greater 
than for their respective control counterparts. 


Additional analyses were conducted to assess whether there was a differential impact of the Striving 
Readers program on the reading performance of different NCLB subgroups among sixth-grade Tier 3 
students who had been offered the intervention services for either one year, two years, or three years. 
For these analyses, NCLB subgroup variables were entered into the models as covariates and their 
interactions with treatment were explored. Analyses were carried out for Analytic Groups 6, 8, and 10 
respectively. Corresponding results are reported in Tables R-10 and R-11, Tables R-16 and R-17, and 
Tables R-22 and R-23 in Appendix R, which present results for the full models and for the final models. 
For all these interaction models, the overall treatment effect was not statistically significant, which is 
consistent with the findings from the corresponding main effects models. 


For Analytic Group 6, sixth-grade Tier 3 students who had been offered the intervention services for 
one year, the interaction between free and reduced-price lunch (FRL) eligibility and treatment in the final 
model was significantly negative, indicating that there were differential program impacts on students of 
different socioeconomic status. Specifically, a differential program impact was found for students who 
were eligible for FRL compared with those who were not eligible. However, it should be noted that the 
“differential impact” might only reflect a difference in performance when comparing FRL-eligible 
students vs. FRL-ineligible students (i.e., the difference between treatment and control students was 
greater for FRL-ineligible students than for FRL-eligible students). It still would not demonstrate that 
the program was “effective” for FRL-ineligible students, because the average performance of the treated 
FRL-ineligible students was not statistically different from that of FRL-ineligible students in the control 
schools.” 


Regarding Analytic Group 8, sixth-grade Tier 3 students who had the opportunity to participate for 
two years, the interaction between English language learner (ELL) status and treatment in the final 


°3 Main treatment effects were tested for each relevant subgroup using additional HLM analyses. None of these 
treatment effects were statistically significant. 
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model was negative and approaches statistical significance (6 = 0.052). This result hints at the possibility 
that the intervention services had a differential impact on non-ELL students and ELL students that was 
not detectable by the current evaluation design. Nevertheless, even a statistically significant “differential 
impact” might only reflect a difference in performance when comparing non-ELLs vs. ELLs (i.e., the 
difference between treatment and control students was greater for non-ELLs than for ELLs); it still 
would not demonstrate that the program was “effective” for non-ELL students, because the average 
performance of non-ELLs in the treatment group was not statistically different from that of non-ELLs 
in the control group.” 


For Analytic Group 10, sixth-grade Tier 3 students who had been provided with three years of the 
intervention services, the interaction between gender and treatment in the final model was significantly 
positive, indicating that there were differential program impacts on boys and girls. Specifically, this 
“differential impact” might only indicate a difference in performance when comparing girls vs. boys (Le., 
the difference between treatment and control students was greater for girls than for boys). It still would 
not demonstrate that the program was “effective” for girls, because the average performance of the girls 
in the treatment group was not statistically different from that of girls in the control schools.” 


No other interactions between treatment and NCLB subgroups were statistically significant for the 
three analytic groups, indicating that there was no evidence that the program has had a differential 
impact on any of these subgroups of students. 


°4 Female students were coded as 1, while male students served as the reference group. 
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Appendix A-1: Year 4 Measures 


This Appendix includes copies of the following surveys, instruments, and protocols used in the 
evaluation of the Chicago Striving Readers program: 


e Surveys 
oO Spring 2010 Literacy Improvement Survey for Teachers — Treatment Schools 
oO Spring 2010 Survey of Literacy Intervention Teachers 
oO Spring 2010 Literacy Improvement Survey for Content Area Teachers — Treatment 
Schools 
oO Spring 2010 Literacy Improvement Survey for Teachers — Control Schools 
oO Spring 2010 Literacy Improvement Survey for Content Area Teachers — Control Schools 
e Interview Protocols: 
Principal Interview Protocol — Treatment Schools 
Principal Interview Protocol — Control Schools 
Project Director Interview Protocol 
Literacy Consultant (Senior Literacy Advisor) Interview Protocol 
District Coordinator Interview Protocol 
District Technology Coordinator Interview Protocol 
ase Study 
Observation Protocol 
Self Contained Teachers Focus Group Protocols (Fall ‘09 & Spring ‘10) 
ELA Teachers Focus Group Protocols (Fall ‘09 & Spring ‘10) 
Non-ELA Teachers Focus Group Protocols (Fall ‘09 & Spring ‘10) 
LIT Interview Protocols (Fall ‘09 & Spring ‘10) 
Principal Interview Protocols (Fall ‘09 & Spring ‘10) 
Spring ’10 Self Contained Teachers Focus Group Protocols for new case study schools 
Spring ’10 ELA Teachers Focus Group Protocols for new case study schools 
Spring ’10 Non-ELA Teachers Focus Group Protocols for new case study schools 
Spring ’10 LIT Interview Protocols for new case study schools 
Spring ’10 Principal Interview Protocols for new case study schools 


oooo0o0o0o0 0000 R000 000 
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Chicago Public Schools (CPS) Striving Readers 
Spring 2010 Literacy Improvement Survey for Teachers 


The following is a survey designed to gather your feedback on the essential components of the Striving Readers 
program. Survey results will be reported in the aggregate only. We will not use your name or identify individual 
respondents. Your feedback is extremely valuable to the success of this program. If you have questions about this 
survey, please contact Rebecca Swann at rswann@metisassoc.com or 212-425-8833. 


1. What is your primary role or teaching assignment? 


(Select the single best option. 


If you are a Literacy Intervention Teacher (LIT), please select that role even if you also have other duties. 
Literacy Intervention Teacher 


General education teacher (self-contained classroom teacher) (Continue with LIS) 
English language arts teacher 
Teach English language arts and other academic subject 0 
Teach other academic subjects but notEnglish language arts 
Bilingual/ELL teacher 
Special education teacher 
Reading specialist 
Other (please specify): 
Does this role include teaching of English language arts? 
e §6Yes 
e No (Jump to “Thank you for completing this survey!”) 


Comprehensive Instruction 


2. Ina typical classroom, how often do you use the following practices to help students increase reading 


comprehension? 
Less 1-3 
; : than | times au is 
Use of Instructional Practices Never Gncea ‘ times a | timesa 
month | month WEEN aaa 

Explicit instruction in the use of any one or more of the 
following comprehension strategies: summarizing, 

Hist aaa Paria 0 0 im) 0 Oo 
questioning, predicting, text structurevisualization, 
inferring and metacognition 
Establishing the purpose for reading. Oo 0 im) 0 0 
Monitoring students’ comprehension through questioning. i) Oo im) 0 0 
Making connections to background knowledge. 0 0 im) 0 0 
Making connections between texts. 0 0 o o o 
Synthesizing information within text or across texts. o o o Oo Oo 
Using differentiated instruction (i.e. providing different content, 
resources and/or instructional techniques specifically tailored to q q qo q q 
meet students’ individual educational needs and/or learning 
styles) 
Use of before, during, and after (BDA) reading strategies for 
comprehension instruction (A student constructed mental 
framework for reading begun before reading even begins, i) i) im) 0 0 
strengthened as students interact with the text during the 
reading, and reflected upon after reading.) 
Using Pairing for Partner Reading & Content Too (PRC2) for q q qo q q 
comprehension instruction 


3. Ina typical classroom, how often do you use the following practices to help students build their vocabulary 


knowledge? 

. ; i ore ae ia 

Use of Instructional Practices Never ances a times a | timesa 
month | month is sia 

Explicit instruction in vocabulary o 0 im) 0 0 
Modeling the use of word parts i) Oo im) 0 0 
Review of vocabulary words 0 0 o 0 o 
Use of vocabulary notebooks 0 0 o im) o 


Less 1-3 
than | times as oe 
Use of Instructional Practices Never times a | timesa 
oncea a 
week week 
month | month 
Use of the PRC2 for vocabulary development. i) 0 0 0 0 
Use of before, during, and after (BDA) reading strategies for q q q q q 
vocabulary instruction 
Words Their Way 0 0 0 0 i) 
Academic Vocabulary for content terms (e.g., Marzano) 0 i) 0 0 Oo 
Word study sorts and concepts (e.g,, Donald Bear) 
Morphology instruction (e.g,, Shane Templeton) 
4. Ina typical classroom, how often do you use the following practices to help students develop fluency? 
than | times | 2? | 48 
Use of Instructional Practices Never timesa | timesa 
once a a 
week week 
month | month 
Teacher read aloud 0 0 0 0 0 
Teacher interactive read aloud 0 0 0 0 0 
Shared reading (students and teacher take turns in reading) 0 0 0 0 0 
Modeling reading for students 0 0 0 0 0 
Explicit instruction in guided oral reading im) im) 0 0 0 
Students listen to audio books, play aways im) 0 0 0 0 


5. Ina typical classroom, how often do you use the following techniques to help students develop better reading 


strategies and skills? 


Techniques 


Never/ Not 
Familiar 


Less than once 
a month 


1-3 times a 
month 


1-3 times a 
week 


4-5 times a 
week 


Everybody Reads To (ERT) 


o 


oO 


im) 


mi) 


Oo 


Exclusion Brainstorming 


List-Group-Label 


Predict-Locate-Add-Note (PLAN) 


ReQuest 


Interactive Notation System for Effective Reading and 
Thinking (INSERT) 


ABC Graffiti 


Guided Reading and Summarizing Procedure (GRASP) 


KWL 


O)/QO;0O);) OQ ;O/;O;0;0 


O)/QO/;0O); OQ ;O/;O/;0;0 


QOQ)O;O; OQ ;QO;QO;0);0 


QOQ),O;O; OQ ;Q;QO;0);0 


Q)O,;O; Q ;QO;0O;0);0 
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6. _ Ina typical classroom, how often do you use the following grouping structures? 


. reso wan 1-3timesa | 1-3timesa | 4-5 times Multipie 
Grouping Structures Never oncea times a 
month week a week 
month day 
Whole class/Large group o o o o 0 0 
Individual Work o o o o 0 0 
Small groups or Pairs o o 0 0 i) o 


7. Considering your own instruction (not that of the LIT or other instructors in your classroom), in a typical classroom, 
how often do you apply differentiated instruction (providing different content, resources and/or instructional 
techniques specifically tailored to meet students’ individual educational needs and/or learning styles) in your 


classroom? 
Almost 
: About half Most of every 
Nevel pA Scrasonally the time the time | lesson or 
activity 
im) 0 0 im) 0 0 


Purposeful Assessment 


8. _ Indicate how you use the data from the following assessments. (Please check all that apply.) 


Assessments 


Not Using 


Screening 


Diagnostic 


Progress 


Benchmarking 


Monitoring 


Assess 
Outcomes 


Reading Benchmark Assessment 


0 


Oo 


0 


og 


oO 


0 


\llinois Standards Achievement 
Test 


Basic Reading Inventory (BRI) 


Informal assessments 


Fluency Snapshots 


Spelling Inventories 


Other: 


Other: 


Other: 


OQ,Oy;Qo,;O;0;0);0; a 


O)O;O)QO;0;0;0; a 


O,Oy;oO,;O;0O/;0);0; a 


OQ)OQ;QO),0;O0/;0;0) a 


O)/O;O)/Q;0;0;0; a 


O,aO;oO,;O;0O/;0);0; a 


Data-Driven Instruction 


9. Please indicate the extent to which you use student assessment data for each of the following purposes. 


Use of Data 


Not at All 


To Some extent 


To a Moderate 
Extent 


To aLarge Extent 


Placing students in intervention programs. 


im) 


Oo 


Oo 


im) 


styles)e 


Differentiating instruction (i.e. providing 
different content, resources and/or 
instructional techniques and materials 
specifically tailored to meet students’ 
individual educational needs and/or learning 


retaught. 


Identifying skills that need to be taught or 


Use of Data Not at All To Some extent ioe To a Large Extent 
Extent 

Monitoring student reading progress. 0 0 0 0 

Creating instructional groups (in-class). 0 im) 0 0 


Grade-Level Teams 


10. 


subject areas from the same grade, or in grade level “bands”.) 


o Yes 
o No (Ifno, skip to Question 13) 


11. 


Overall, rate the grade-level team’s ability to use classroom assessment data in the following ways. 


Do you currently have grade-level teams at your school? (Grade-Level Teams are teams consisting of staff across 


Use of Data 


Poor 


Fair 


Good 


Excellent 


Not Sure 


Address the literacy needs of all students. 


a) 


o 


Oo 


Oo 


Oo 


Address the needs of struggling readers. 


Formalize lesson plans. 


Identify students who are eligible for targeted 
interventions. 


Identify strengths. 


Identify teaching and learning strategies. 


Improve classroom practice. 


QOQ)/O)O); OQ ;O/;O 


OQ;/O)O); @ ;O/;0 


Q;Q;Q0; Q ;oO;a 


OQ;O/;Q; OQ ;oO;a 


OQ)/O,)O,; OQ ;QO/;u 


Literacy Teams 


12. Do you currently have a literacy team in place at your school? (A literacy team is a team that focuses on literacy 


issues across grade levels.) 
o Yes 
0 No (If no, skip to Question 15) 


13. Overall, rate the quality of the literacy team’s performance in the following areas. 


Performance Areas 


Poor 


Fair 


Good 


Excellent 


Not Sure 


Using assessment data to pinpoint the staff's 
professional development needs. 


im) 


o 


a) 


mi) 


o 


Addressing the needs of all students. 


Addressing the needs of struggling readers. 


Addressing the needs of grade-level teams. 


Improving literacy instruction at your school. 


O;O);O);a0 


O}O)O);0 


O};O)O);0 


O};O)O);0 


O};O)O);0 


School-wide Intervention Materials 


14. For each of the materials listed below, 
Indicate how frequently you currently use the materials to teach literacy in your typical classroom. 


For those that you are using, rate how comfortable you are with using these materials to support student learning in language arts. 


ie b) IF USING: 
Please rate your confort level 

Materials NA Not Lessthan | 1to3 1to3 A4to5 1 2 3 A: 5 

(Do Not } Currently oncea times a times a times a Not at all Very 

Have) Using month month week week Comfortable Comfortable 
Listening centers (Classroom CD 
sa aS aaa o o o fm) o o o Oo o oO fm) 
headphones) 
brehas Giese o o o oO o o o Oo Oo Oo oO 
Classroom library o o o o o o o 0 o o o 
Vocabulary notebooks oO Oo oO o o Oo | Oo | Oo Oo 
Reading response notebooks o o o o o o o o oO 0 Oo 
Schoo library og o oO og Oo Oo | Oo | Oo Oo 
Reading Anthologies og o Oo og Oo oO | o | Oo Oo 
Reading Basals oO o Oo o Oo Oo | Oo | Oo Oo 
mea ae bd ale oO Oo o Oo oO o o Oo o o o 
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15. For each of the raterials listed across the top of the chart below, please indicate which literacy instructional goals are supported by your use of that material in your 


classroom. (check all that apply.) 
Listening Classroom Readingbasals School 
Instructional goals that each material is centers Media centers library L Library 
used to support: J J J 
O o a = a a = 


Materials: 


e 
eo 


Vocabulary Reading 
notebooks anthologies 
1 
Oo 


Vocabulary Development 
Grammer 


|Gammer 


To develop students’ self-directed learning 
To supplement students’ textbook reading 


Teaching students to identify and use text 
structure 


Teaching students to identify and use the 


organizational features of expository writing 
To activate students’ prior knowledge 


o 
o 


Oo 
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16. For each of the materials listed below, please indicate whether the following grouping strategies or differentiated 
instruction are supported by your use of these materials in your classroom. (Check all that apply) 


Materials 


Not 
Using 


Instructional Groupings and Differentiation 


Whole Class/ 
Large Group 


Individual 
Work 


Small Group/ 
Pairs 


Support 
Differentiated 
Instruction 


Listening centers 


Q 


Q 
Q 


Oo 


Media centers 


Text sets 


Classroom library 


Vocabulary notebooks 


Textbooks 


Reading response notebooks 


School Library 


Reading anthologies 


Readingbasals 


Other informational texts (other than text sets) 


gqQayqgaggaggqngagaaaa 


gqgagqgaggqgggagaua 


gqayagqgaggagagqgnagqgagqguaa 
gqagqgagaggqgqgnagqgaga 


gqagqgaggagnagqaqgugaa 


Use of Handheld Computers 


17. a) Do you use handheld computers (Palm Pilots) to teach literacy? 


Yes (Skip to Question 19.) 
No 


b) If you are NOT yet using handheld computers, please indicate why you are not using them below (Check all 


that apply) and then SKIP to Question 24: 

| have not received the handheld computers. 
Some or all of the computers are not working properly. 
Some or all of the necessary software applications have not been installed on the computers. 
| have not received sufficient professional development to feel comfortable using them. 


Because the Striving Readers program provides only 10 computers per classroom, and | do not like to have 
some students use them while others can not. 


| do not feel that they offer sufficient added benefit compared to traditional media (e.g. print, paper and 


pencil) to be worth the trouble. 
Other (please specify): 


18. In atypical classroom, how often do your students use handheld computers (Palm Pilots) during literacy 


instruction? 

o Less than once a month 
o  1-3times amonth 

o 1-3 times a week 

o 4-5 times a week 
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19. Rate how comfortable you are with using the Palm Pilotsto support your literacy instruction 


1 2 3 4 5 
Not at all Comfortable Very Comfortable 
Oo Oo Oo og ) 


20. 


21. 


Which specific academic foci or instructional objectives do you support with the use of handheld computers (Palm 
Pilots)? (Check all that apply) 


Fluency 

Vocabulary development 

Developing students’ reading comprehension strategies 
Writing skills 

Word parts 

Word recognition 

Spelling 

Grammar 

Locating information 

Evaluating information 

Synthesizing information 

Organizing information 

To develop students’ self-directed learning 

Teaching students to identify and use the organizational features of expository writing 
To activate students’ prior knowledge 


Which of the following instructional activities and practices do you support with the use of handheld computers 
(Palm Pilots)? (Check all that apply) 


Whole class/ Large group 

Small group/ Pairs 

Individual Work 

Monitoring distribution and completion of assignments 
Assessing students’ literacy skills 

Monitoring students’ progress 


Differentiating instruction (i.e. providing different content, resources and/or instructional techniques 
specifically tailored to meet students’ individual educational needs and/or learning styles) for struggling 
readers (Tiers 2 & 3) 


Differentiating instruction for English language learners/special education students 
Guided reading 

Partner reading 

Individual reading 

Book club discussions 
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22. Inthe table below, please indicate your frequency and comfort of use of each software application when using the handheld computers for literacy instruction. 


a) 
b) 


Indicate how frequently students in your typical classroom currently use each software application on the Palm Pilots during literacy instruction. 
For those that you are using, rate how confortable you are with using each software application on the Palm Pilotsto support your literacy instruction. 


Handheld Computer Software 


Applications 


a) Frequency 


b) IF USING: 
Rate your comfort level 


Less than 
once a 


1to3 
timesa 
month 


3 


Q 


QO; O aj gjafafafafafajajajalé 


OQ;Q  ; ggqga;agyoy;yo;QO/;0);07;0/0 


OQ} ao ;Q,O),0;/0/0/0/0/0/0/0/0 


O;,Q go, gQ/0;/0)/0;/0/0);0/;0/0 


O,ag;s;QQ00/O0O;/0/0;/0/;/0/0/0/0 


O;,aQ ggaayo;yao;o;0;0)0 


OQ;Q gq gQ0;/0;/0)/0;/0/0)/0/;0/0 


QOQ;,Q gq gQag;/0;/0)0;/0/0;0/;0/0 


OQ;,Q/ gq gag;/0;/0)/0;/0/0);0/;0/0 


QOQ;,Q gq gag;/0;/0)/0;/0/0);0/;0/0 


ol]a 2 aala/o}alala/a}alal ig 
() 
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Classroom Library 


23. Please check the ways that you use your classroom libraries. (Check all that apply) 
1 For content area instruction 
[1 For independent reading 
11 For small group instruction 
[1 For read alouds 
24. Do you use interest inventories to help students self select reading material? 
o Yes 
o No 
25. Do you use interest inventories to guide your purchases for the classroom library? 
o Yes 
o No 
26. Please indicate how true each of the following statements are about the organization of books in your classroom library. 
Not At All Slightly Somewhat V 
. ery True 
My classroom library... True True True 
..IS easily accessible to students. 0 0 i) im) 
...IS well organized and in good shape. Oo im) 0 0 
...has a checkout system in place. 0 0 0 0 
... includes a variety of reading materials that are 
en i 0 Oo 0 0 
appropriate for readers of differing abilities. 
.includes a variety of texts that appeal to readers with q qo q q 
differing interests. 
...has reading materials grouped by genre. o o o o 
...has reading materials clearly labeled. o o o o 
..has both nonfiction and fiction books. qo oO oO Oo 


27. 


28. 


To what extent are you able to consider students’ needs and reading abilities when ordering books and other reading 
material with Striving Readers funds for your classroom library? 

o Notatall 

To asmall extent 

To a moderate extent 

To a large extent 

Don’t know 


Oo0o0°0 


To what extent are you able to consider students’ interests and motivation when ordering books and other reading 
material with Striving Readers funds for your classroom library? 

o Notatall 

To a small extent 

To a moderate extent 

To a large extent 

Don’t know 


oo0o0°o 


School Library 


29. 


In a typical classroom, how often do you take your class to the library? 
oOo Never 

Rarely (less than once a month) 

Sometimes (at least once a month) 

Often (at least once a week) 


oo°o 
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O 


Almost daily or daily 


30. To what extent do the library resources support the Striving Readers program? 


Oo 


Ooo0°0 


Not at all 

To asmall extent 

To a moderate extent 
To a large extent 
Don’t know 


31. Do you have a school librarian? 


Yes 
No [Skip to question 35] 


32. How does the librarian work with you? (Check all that apply.) 


The librarian does not work with me. 

The librarian provides resources for class projects. 

The librarian and | collaborate on how to supplement lessons with library resources. 
Other (please specify): 


33. To what extent does the librarian consult with classroom teachers in using Striving Readers library funds to order reading 
materials that are grade level and content appropriate? 


Not at all 

To asmall extent 

To a moderate extent 
To a large extent 
Don't know 


34. How does the librarian work with your students? (Check all that apply.) 


io} 


Does not work with my students. 

Works with students on research skills. 

Directs students to resources tied to curriculum. 

Conducts read-alouds. 

Provides students with information about extracurricular academic activities (e.g., spelling bee, writing competitions, 
events). 

Assists students with class projects. 

Teaches students how to navigate Internet resources. 

Guides struggling readers to summer programs. 

Other (please specify): 


Collaboration with LIT 


35. How often do you meet or collaborate with the LIT in the following settings? 


Meeting Structures Never Less than 1-3 times a 1-3 times a 4-5 times a 
once a month month week week 

Scheduled one-on-one meetings Oo 0 i) 0 0 

Impromptu one-on-one meetings (during lunch, q q q q q 

prep periods, before/after school, etc.) 

Grade-level team meetings o o o oO o 

Literacy team meetings o o o oO o 
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36. To what extent has your collaboration with the LIT facilitated your efforts to use the following methods to support struggling 
readers in your class? 


Instructional methods 


Extent to which collaboration with LIT facilitated use of methods 


Not at all 


Toasmall 
extent 


To a moderate 
extent 


To a large 
extent 


Differentiating instruction 


im 


og 


oO 


0 


Scaffolding of instruction 


Student groupings 


Using the media center 


Using listening centers 


>|olalo|o)/» 


Using handheld computers 


Using assessment data to monitor student progress 


> |e 


Using student assessment data for instructional planning 


Q)O;QO;0/;O/;0);0 


O;O);O/;O;0;0)0 


O;)/QO;0O/;0)0/;0/;0 


Q;OQ;Q0;O0/O);0);0 


37. To what extent has your collaboration with the LIT facilitated your ability to provide effective instruction in the following areas 
for struggling readers? 


Academic areas 


Extent to which collaboration with LIT facilitated effective instruction 


Not at all 


Toasmall 
extent 


To a moderate 
extent 


To a large extent 


a. Comprehension 


im) 


mi) 


im) 


mi) 


b. Fluency 


c. Vocabulary 


d. Writing skills 


e. Word parts 


f. Word recognition 


g. Spelling 


h. Reading/literacy in content areas 


OQ;O;o;O;0);07;0 


O);O);O);0O/;0;0)0 


O;O);O);0O/;0;0)0 


O);O);O);0O/;0;0)0 


38. Overall, how effective has the literacy intervention teacher (LIT) push-in been in improving the reading skills of struggling 


readers in your classroom? 


O 


oo0o°0 


Not at all effective 
Minimally effective 
Somewhat effective 
Effective 

Very effective 


Professional Development 


39. For each of the following Striving Readers professional development sessions conducted during the 2009-2010 school year, 


please indicate: 


= Whether you participated, and 
= _ If so, how useful the session(s) was (were) in helping you support student learning in language arts 


Did Ou If YES, how useful was the session? 
, : participate? 
Professional Development Sessions 
No Yes Not Somewhat | Moderately | Extremely 
Useful Useful Useful Useful 

AMP Intensive Intervention Program Training 0 i) 0 0 0 o 
2009 Summer institute 0 i) 0 0 0 o 
School-year follow-up institutes 0 0 0 0 0 o 
Technology training (use of handhelds) i) 0 0 0 0 o 
Training in LIT/ teacher collaboration Oo 0 0 0 0 im) 
School-based professional development 0 0 0 im) 0 o 


40. For each of the following topics, indicate: 


= Whether you received professional development addressing this topic during the current year 
= If so, rate the impact that professional development you received has had on your comfort with each teaching practice. 


Teaching practices 


Received PD? 


If YES, what impact did the professional development have on your comfort 


with each teaching practice? 


i 


Slight Impact 


Moderate Impact 


Major Impact 


Building academic vocabulary 


o 


og 


im) 


Classroom libraries 


Creating literacy-rich classroom environments 


Differentiating instruction 


Explicit vocabulary instruction 


Increasing student motivation 


Supporting students’ self-directed learning 


Using before, during, and after reading strategies and techniques 


Using student assessments to guide and inform instruction 


Using handheld computers (Palm Pilots) for teaching and learning 


Using literacy-based software 


Using the PRC2 model 


Using the whole-part-whole classroom instruction model 


O;O/0/0/0;0)/0)/0/0)0)/0/0/0/¢ 


OQ)/O,Q;O;/0;/0/;/0/0/;0/0/;0;0;0 


5 
i iad cde Fs Res Pa 


OQ;O;O;O;/0)/0;0/;0/;0;0);0 


Q)O,Q;QO;/O0)/0/;/0;/0/;0;0);0/;0 


O;O;O;O;/0);0;0/;0/;0;0);0 
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aS 
rary 


DOOUOOOOOOVOVOUOUKOUOOUOUD OCOUOOUUOUD: 


Please check the techniques in the list below for which you would like to receive more training. (Check all that apply) 
Academic Vocabulary for content terms (e.g., Marzano) 

Morphology instruction (e.g,, Shane Templeton) 

Word study sorts and concepts (e.g,, Donald Bear) 

Words Their Way(e.g,, Donald Bear & Shane Templeton) 

KWL 

Using PRC2 for comprehension instruction. 

Using PRC2 for vocabulary development. 


Differentiating instruction (i.e. providing different content, resources and/or instructional techniques specifically tailored to 
meet students’ individual educational needs and/or learning styles) 


Everybody Reads To (ERT) 

Exclusion Brainstorming 

List-Group-Label 

Predict-Locate-Add-Note (PLAN) 

ReQuest 

Interactive Notation System for Effective Reading and Thinking (INSERT) 
Read Aloud/Think Aloud 

ABC Graffiti 

Guided Reading and Summarizing Procedure (GRASP) 
Teaching summarizing as a comprehension strategy 
Teaching questioning as a comprehension strategy 
Teaching predicting as a comprehension strategy 
Teaching text structure as a comprehension strategy 
Teaching visualization as a comprehension strategy 
Teaching inferring as a comprehension strategy 


Teaching metacognition as a comprehension strategy 


Struggling Readers: Extended Day (Afterschool) Intervention 


42. \n addition to English language arts, what other subject areas do you teach? 
Mathematics ( 
Science ( 
Social studies ( 
Other: (Please Specify: ) 


About You [Alll types of respondents] 


43. What is the name of your school? 
[drop down list] 


po HENDRICKS 
BEETHOVEN HENSON 
BETHUNE LINNE 


BURR LOVETT 
BURROUGHS MANIERRE 


GRAY 


44, 


45, 


46, 


47, 


48, 


At which grade level(s) are you teaching reading/English language arts this year (2009-10)? (Check all that 
apply): 


0k010203040506070809 010 011 012 


In which of the following settings do you teach literacy? (Check all that apply) 
oO Self-contained 

Subject-Area specialist 

Departmentalized 

Double block 

Other (Please specify): 


oo0°0 


How many years have you been teaching? [INSERT TEXTBOX] 
How many years have you been teaching at this school? [INSERT TEXTBOX] 


How many years have you been teaching reading? [INSERT TEXTBOX] 


Teachers will be reimbursed by CPS-Striving Readers for their time to complete this survey. In order to be reimbursed we need 
you to identify yourself so that we can verify that you completed the survey. If you would like to be reimbursed, please provide 
your name and email address below, and be sure that you identified your school in the previous item. Your survey responses 
will still remain strictly confidential and will never be reported in any form that would allow anyone to connect your responses with 
your name. Providing this information is optional. 


Your Name: Email: 


Thank you for completing this survey! 


Chicago Public Schools Striving Readers 
Spring 2010 Survey of Literacy Intervention Teachers 


The following is a survey designed to gather your feedback on the push-in intervention and AMP after-school components of 
the Striving Readers program for Tier 2 and 3 students. It will take you approximately 60 minutes to complete (approximately 
30 minutes for AMP-only teachers). Results will be reported in the aggregate only; we will not use your name or identify 
individual respondents. Your feedback is extremely valuable to the success of this program. If you have questions about this 
survey, please contact Rebecca Swann at rswann@metisassoc.com or 212-425-8833. 


Please answer the following questions with regard to your work with students in the Targeted intervention 
group (i.e., additional instruction in small group setting for Tier 2-3 students). 


1. Ina typical classroom, how often do you use the following grouping structures? 


: Eess walt 1-3timesa | 1-3timesa | 4-5 times Multiple 
Grouping Structures Never once a times a 
month week a week 
month day 
Whole class/Large group o o o i) Oo 0 
Individual Work o o o o i) Oo 
Small groups or Pairs 0 o 0 0 0 oO 


2. Considering your push-in intervention with Tier 2and 3 students, in a typical classroom, how often do you apply 
differentiated instruction (providing different content, resources and/or instructional techniques specifically 
tailored to meet students’ individual educational needs and/or learning styles)? 


Almost 
: About half Most of every 
Never Rae Occasional) the time the time | lesson or 
activity 
0 0 0 0 0 0 


3. Duringyour work in the regular classroom with students in the Targeted intervention group (Tier 2 and 3 
students), in a typical classroom, how often do you use the following practices to help struggling readers 


increase reading comprehension? 


Less than 1-3 1-3 4-5 
Use of Instructional Practices Never | oncea | timesa | timesa | timesa 
month month | week week 

Explicit instruction in the use of any one or more of the following 
comprehension strategies: summarizing, questioning, predicting, text 0 0 0 Oo 0 
structurevisualization, inferring and metacognition 
Establishing the purpose for reading 0 0 im) Oo 0 
Monitoring students’ comprehension through questioning 0 0 0 im) 0 
Making connections to background knowledge 0 Oo Oo 0 o 
Making connections between texts 0 Oo Oo o o 
Synthesizing information within text or across texts 0 Oo 0 oO 0 
Using differentiated instruction (providing different content, resources and/or 
instructional techniques specifically tailored to meet students’ individual 
educational needs and/or learning styles). 
Use of before, during, and after (BDA) reading strategies for comprehension Oo oO Oo Oo oO 


Less than 1-3 1-3 4-5 
Use of Instructional Practices Never once a times a | timesa | timesa 
month month | week week 
instruction (A student constructed a mental framework for reading begun before 
reading even begins, strengthened as students interact with the text during the 
reading, and reflected upon after reading.) 
Using Pairing for Partner Reading & Content Too (PRC2) for comprehension q q q q q 
instruction 
4. Duringyour work in the regular classroom with students in the Targeted intervention group (Tier 2 and 3 
students), in a typical classroom, how often do you use the following practices to help struggling readers build 
their vocabulary knowledge? 
Less than 1-3 1-3 4-5 
Use of Instructional Practices Never once a times a | timesa | timesa 
month month | week week 
Explicit instruction in vocabulary 0 0 0 Oo Oo 
Modeling the use of word parts 0 0 0 Oo 0 
Review of vocabulary words Oo Oo 0 Oo Oo 
Use of vocabulary notebooks Oo Oo Oo o o 
Use of the PRC2 for vocabulary development 0 0 0 0 Oo 
Use of before, during, and after (BDA) reading strategies for vocabulary q q q q q 
instruction 
Words Their Way 0 0 Oo Oo Oo 
Academic Vocabulary for content terms(e.g. Marzano) o o o o o 
Word study — word sorts and concepts (e.g. Bear & Templeton) o o o o o 
Morphology instruction(e.g. Bear & Templeton) o o Oo o o 
5. Duringyour work in the regular classroom with students in the Targeted intervention group (Tier 2 and 3 
students), in a typical classroom, how often do you use the following practices to help struggling readers develop 
fluency? 
Less than 1-3 1-3 4-5 
Use of Instructional Practices Never once a times a | timesa | timesa 
month month | week week 
Teacher read aloud 0 Oo Oo 0 0 
Teacher interactive read aloud Oo Oo 0 Oo 0 
Shared reading (students and teacher take turns in reading) 0 Oo o o o 
Modeling reading for students Oo Oo 0 o o 
Focusing instruction on proper and meaningful phrasing 0 im) 0 i) 0 
Students listen to audio books, play aways Oo 0 Oo Oo o 
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The gradual release model (Leading students from modeled instruction to shared instruction to guided practice 


and finally students’ independent practice) and explicit instruction in guided reading are intended to be use on an 


“as needed” basis. 


Duringyour work in the regular classroom with students in the Targeted intervention group, in a typical 
classroom, to what extent do you feel you are able to meet your Tier 2 and 3 students’ individual needs through 


these instructional practices? 


Extent the technique meets 
Not Tier 2 and 3 Students’ needs in this area 
Instructional Practices and Purposes : Toa 
Using To some To a large 
Not at all Axton moderate etant 
extent 

Use of the gradual release of responsibility model q q q q q 
for reading comprehension instruction 

Use of the gradual release of responsibility model q q q q q 
to build vocabulary 

Use of the gradual release of responsibility model q q q q q 
to develop fluency 

Explicit instruction in guided oral reading q q q q q 
to develop fluency 


7. Duringyour work in the regular classroom with students in the Targeted intervention group (Tier 2 and 3 
students), in a typical classroom, how often do you use the following techniques to help students develop better 
reading strategies and skills? 

Tetniques Pe een ee ee |e 
Everybody Reads To (ERT) i) 0 0 Oo 0 
Exclusion Brainstorming Oo 0 Oo o o 
List-Group-Label 0 i) 0 Oo 0 
Predict-Locate-Add-Note (PLAN) 0 0 0 Oo 0 
ReQuest Oo 0 Oo o o 
aad Nee System for Effective Reading and q q q q q 
ABC Graffiti 0 0 0 0 0 
acer and Summarizing Procedure q q q q q 
KWL 0 im) 0 0 0 


8. How often do you meet with ELA classroom teachers at the following grade levels to discuss instruction-related 
issues regarding your work with students in the Targeted intervention group? 


Frequency of Meetings with Grade-Level Teachers 


Less than 1-3 1-3 4-5 
Never oncea | timesa | timesa | timesa 
month month week week 
Grade 6 teachers | 0 | | | 
Grade 7 teachers 0 0 | | | 
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Grade 8 teachers 


Oo 


o 


Oo 


Oo 


o 


9. 


In which setting(s) do you typically meet or collaborate with ELA classroom teachers? (Check all that apply for 


each grade) 


Settings for Meetings with Classroom Teachers Grade 6 Grade 7 Grade 8 
a. Scheduled one-on-one meetings 0 0 0 
b. Impromptu one-on-one meetings (é.g., during lunch, prep 
0 0 0 
periods, before/after school, etc.) 
C. Grade-level team meetings (i.e.,teams consisting of staff 
across subject areas from the same grade, or in grade level 0 Oo o 
“bands”) 
d. Literacy team meetings (i.e.,teams focusing on literacy 
0 0 0 
issues across grade levels.) 
10. On average, how often do you meet with eachSIXTH-GRADE classroom teacher to discuss implementing each 


of the following instructional methods for students in the in-class Targeted Intervention group (Tiers 2-3)? 


How often discussed with classroom teachers 


Instructional methods and activities for Targeted Less than ; : : 
Intervention Never Glicea® IVa eee | Se mees oMMes 
a month a week a week 
month 

Differentiated instruction 0 Oo 0 0 0 
Student groupings Oo Oo Oo Oo im) 
Use of Striving Readers texts sets, text set teacher 7 q 4 q 
guides, technology, classroom library, school library 
Use of specific Striving Readers comprehension q 7 q 4 q 
strategies for reading 
Using specific Striving Readers instructional q 7 q 4 q 
techniques for comprehension instruction 
Using specific Striving Readers instructional 5 5 5 5 q 
techniques for vocabulary instruction 
Using specific Striving Readers instructional 
techniques for fluency instruction 
Discussing specific students’ reading progress 
Coordinating instruction between lessons for the 
whole class and lessons for the Targeted Intervention 0 0 0 o o 
group 
Using student assessment data for instructional 4 7 q q q 
planning 
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11. Please indicate the extent to which you use student assessment data for each of the following purposes related 


to your work with students in the Targeted intervention group? 


Toa 


Use of Data Not at All Nee moderate vii 
extent 

Differentiating instruction Oo 0 0 o 

Identifying skills that need to be taught or retaught 0 0 Oo 0 

Monitoring student reading progress Oo Oo Oo o 

Creating instructional groups (in-class) 0 0 0 0 
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Schoolwide Intervention Materials 


12. Please use the table below to tell us about your use of various Striving Readers materials during your push-ininstruction of Tier 2 and 3 students in the regular 
classroom. Please rate the use of these materials only in reference to your own instructional activities; do not include those of the classroom teacher or other 
adults who may be assisting in the classrooms you serve. 


For each of the materials listed below, 
= Indicate, in a typical classroom, how often you use the materials to teach literacy. 
=" For those that you are using, rate how comfortable you are with using these materials to support student learning in language arts. 


3 b) IF USING: 
a Please rate your comfort level 
Materials NIA Not Fb 1to3 | 1to3 | 4to5 z = 2 - > 
(Do Not | Currently a timesa | timesa timesa Not at all oy 
Have) Using month week week Comfortable Confortable 
month 
Listening centers (Classroom 
CD & Cassette Player, Read- 
AGreeins NaS Oo 0 Oo Oo 0 0 Oo 0 Oo 0 Oo 
headphones) 
Media centers (fee oq q q oq qo q oq oq qo oq oq 
computers and a printer) 
Classroom library o o o o o o o o o o o 
Vocabulary notebooks o o o o o o o o o o o 
Text sets 0 0 i) Oo Oo 0 Oo 0 0 Oo 0 
Reading response notebooks o o o o o o o o o o o 
School library o o o o o o o o o o o 
Reading Anthologies o o o o o o o o o o o 
Reading Basals 0 0 im) 0 Oo Oo Oo Oo 0 Oo Oo 
Other informational texts oq q oq oq oq oq oq oq oq oq q 
(other than text sets) 
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Use of Handheld Computers during Targeted Intervention Instruction 


13. a) Do you use handheld computers (Palm Pilots) to teach literacy during Targeted Intervention 
instruction of Tier 2 and 3 students in the regular classroom? 


Yes (Skip to Question 14.) 
No (Complete 13b. then skip to Question 21) 


b) If you are NOT yet using handheld computers, please indicate why you are not using them below 
(Check all that apply) (Then SKIP to the After-school Program [AMP] section): 


| have not received the handheld computers. 

Some or all of the computers are not working properly. 

Some or all of the necessary software applications have not been installed on the computers. 
| have not received sufficient professional development to feel comfortable using them. 


Because the Striving Readers program provides only 10 computers per classroom, and | do not 
like to have some students use them while others can not. 


They are being used primarily for whole class instruction. 


| do not feel that they offer sufficient added benefit compared to traditional media (e.g. print, paper 
and pencil) to be worth the trouble. 


Other (please specify): 


14. Ina typical classroom, how often do your students use handheld computers (Palm Pilots) during 
Targeted Intervention instruction of Tier 2 and 3 students? 
o Less than once a month 
o 1-3 times a month 
o 1-3 times a week 
o 4-5 times a week 


15. Rate how comfortable you are with using the Palm Pilotsto support your literacy instruction during 
targeted intervention instruction. 


1 2 3 4 5 
Not at all Very 
Comfortable Comfortable 
0 0 0 0 0 


16. Which specific academic foci or instructional objectives do you support with the use of handheld 
computers (Palm Pilots) during Targeted Intervention instruction? (Check all that apply) 


Fluency 

Vocabulary development 

Developing students’ reading comprehension strategies 
Writing skills 

Word parts 

Word recognition 

Spelling 

Grammar 

Organizing information 
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Locating information 

Evaluating information 

Synthesizing information 

Demonstrate knowledge of key concepts 

To develop students’ self-directed learning 

Teaching students to identify and use the organizational features of expository writing 
To activate students’ prior knowledge 


. Which instructional methods do you support with the use of handheld computers (Palm Pilots) during 


Targeted Intervention instruction? (Check all that apply) 

[1 Monitoring distribution and completion of assignments 
Assessing students’ literacy skills 

Monitoring students’ progress 

Teaching comprehension strategies 

Using comprehension techniques 

Guided reading 

Partner reading 

Individual reading 

Book club discussions 


. Please indicate whether you use handheld computers (Palm Pilots) to support grouping structures 


and/ordifferentiated instructionduring Targeted Intervention instruction of Tier 2 and 3 students in the 
regular classroom, by checking the appropriate strategies below. (Check all that apply) 


[1 Whole class/ Large group 
Small group/ Pairs 
C1 Individual Work 


(| Differentiating instruction (i.e. providing different content, resources and/or instructional techniques 
specifically tailored to meet students’ individual educational needs and/or learning styles) for 
struggling readers 


QO Differentiating instruction for English language learners/special education students 
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19. Inthe table below, please indicate your frequency and comfort of use of each software application when using the handheld computers during your Targeted 


Intervention instruction of Tier 2 and 3 students in the regular classroom. 
a) Indicate how frequently your typical Tier 2-3 students use each software application on the Palm Pilots during literacy instruction. 


b) For those that you are using, rate how comfortable you are with using each software application on the Palm Pilots to support your 
literacy instruction. 


a) Frequency 


b) IF USING: 


Rate your comfort level 


Less than 1to3 
oncea times a 
month 


3 


4 


Q) 


MS Word 


MS Excel 


Slideshow to Go 


Cells 


Internet Browser 


Inspiration 


PAAM management software 


application 


Go Manage 


QO; O |O;JO;JO;,QOQ;Q/;/Q/0/0);0;0;0 


OQ; OQ |O;JO;JO;VQ;Q/Q/Q/0);0;0;0 


QO; OQ |OU;JO;JO;,Q;Q/Q/Q0/0;0;0;0 
OQ; O |O;JO;JO;VQ;Q/Q/Q/Q0);0;0);0 


QO; O |O;JO;JO;,QO;QO/Q0/0/0;0;0;0 


QO; O |O;JO;,O;Q;QO/;/O0/0/0;0)0 


QO; O |O;JOJO;,Q;Q/Q/Q0/0);0;0;0 


QO; OQ |O;JO;JO;,Q;Q/Q/Q0/0);0;0;0 


OQ; OQ |O;JO;JO;VQ;Q/Q/0/0;0;0;0 


QO; OQ |O;JO;JO;VQ;Q/;/QO/0/0);0;0;0 


QO; OQ |jO;JO;,O;Q;QO/O0/0/0;0)0 
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20. For each of the handheld computer software applications listed below, please indicate which literacy instructional objectives are supported by your use of that 
application during your Targeted Intervention instruction of Tier 2 and 3 students. (Please check all that apply.) 


Handheld Computer Software 


Instructional Objectives supported by this application 


Vocabulary 


: 


Reading 


Writing 
Skills 


Word 
recognition 


Spelling 


IKWL 


Oo 


Q) 


Freewrite 


PiCo Maps 


Viewpoint 


Setchy 


MS Word 


MS Excel 


Slideshow to Go 


Cells 


Internet browser 


Inspiration 


qaqqqgaqaqgagqjna ay 


heed le 


gHagqugaaqgagqagqauau 


gHaqgugaaqggqgagagqaau 


gqaaggngaagqgagaaqua 


Q} a} a} a} a} a) a} a} gf of of  & 
jo} 


gHaquwaaagaqagqu 


gHagqgunaaaqgagqaqu 


gqaaggugnagaagqggaggquaau 
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Professional Development 


21. For each of the following Striving Readers professional development sessions conducted during the 2009-2010 
school year, please indicate: 
= Whether you participated, and 
« — Ifso, how useful the session(s) was (were) in helping you support student learning in 
language arts 


Did you Bs 

. . participate? If YES, how useful was the session? 

Professional Development Sessions 
No | Yes Not | Somewhat | Moderately | Extremely 
Useful Useful Useful Useful 

AMP Intensive Intervention Program q q q q q q 
Training 
2009 Summer institute 0 i) 0 0 0 Oo 
School-year follow-up institutes im) 0 im) 0 0 Oo 
Bi-weekly LIT training sessions/ q q q q q q 
Teacher/LIT collaboration 
School-based professional development 0 0 0 0 0 0 


22. For each of the following topics, indicate: 


= Whether you received professional development addressing this topic during the current 


year 


« — If so, rate the impact that the professional development you received has had on your 
comfort with each teaching practice. 


: If YES, what impact did the professional development have 
Received PD? f A : 
: ; on your comfort with each teaching practice? 
Teaching practices 
: Moderate : 
No Yes | Nolmpact | Slight Impact impact Major Impact 
Building academic vocabulary Oo Oo Oo 0 o o 
Classroom libraries 0 0 0 Oo Oo Oo 
Creating literacy-rich classroom q q q q q q 
environments 
Differentiating instruction o o o o o o 
Explicit vocabulary instruction 0 0 0 0 Oo Oo 
Increasing student motivation 0 Oo Oo Oo o o 
Supporting students’ self-directed learning q q q q q q 
(Gradual release model) 
Using before, during, and after reading q q q q q q 
strategies 
Using student assessments to guide 
instruction 
Using handheld computers (Palm Pilots) 
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If YES, what impact did the professional development have 


2 
Received PD? | 4 vour comfort with each teaching practice? 


Teaching practices 


; Moderate ‘ 
No Yes | Nolmpact | Slight Impact impact Major Impact 
Using literacy-based software 0 0 Oo 0 0 0 
Using the PRC2 model Oo Oo Oo Oo Oo Oo 


23. Please check the techniques in the list below for which you would like to receive more training. (Check all that 
apply) 

Academic Vocabulary for content terms(e.g. Marzano) 

Morphology instruction (e.g. Bear &Templeton) 

Word study - word sorts and concepts (e.g. Bear &Templeton) 

Words Their Way 

KWL 

Using PRC2 for comprehension instruction. 

Using PRC2 for vocabulary development. 


Differentiating instruction (i.e. providing different content, resources and/or instructional techniques specifically 
tailored to meet students’ individual educational needs and/or learning styles) 


Everybody Reads To (ERT) 

Exclusion Brainstorming 

List-Group-Label 

Predict-Locate-Add-Note (PLAN) 

ReQuest 

Interactive Notation System for Effective Reading and Thinking (INSERT) 
Read Aloud/Think Aloud 

ABC Graffiti 

Guided Reading and Summarizing Procedure (GRASP) 
Teaching summarizing as a comprehension strategy 
Teaching questioning as a comprehension strategy 
Teaching predicting as a comprehension strategy 
Teaching text structure as a comprehension strategy 
Teaching visualization as a comprehension strategy 
Teaching inferring as a comprehension strategy 
Teaching metacognition as a comprehension strategy 


OUUCVCOOKOOKOOOUOOCOOKCOOOD CUUVOUOODDD 
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Please answer the following questions with regard to your work with students in the AMP after-school program. 


Grouping Structure 


1. Inatypical AMP after-school class, how often do you use the following grouping structures? 


Grouping Structures Never ee aa ae : ye = 
Whole class/Large group Oo 0 0 0 
Individual Work o o o o 
Small groups or Pairs o 0 0 Oo 


2. Ina typical AMP after-school class, how often do you apply differentiated instruction (providing different content, 
resources and/or instructional techniques specifically tailored to meet students’ individual educational needs 
and/or learning styles)? 


Occasionally Usually In every or In every or 
Rarely or Never | (once or twice a (in most nearly every nearly every 
week) lessons) lesson activity 
Oo Oo Oo Oo Oo 


Program Participants 


3. Do you feel that the AMP after-school program is appropriate to the reading levels of the students who are 


currently participating? 

Not at all appropriate 
Somewhat appropriate 
Appropriate 

Very appropriate 


(e) 


ooo 


4. Please indicate the proportion of your AMP students for whom the following statements are true. 


All or 
almost Most About A few pes 
all students | half | students 
enidents students 

These students should not be in the AMP class q q q q q 
because their reading levels are too high. 
These students should not be in the AMP class q q q q q 
because their reading levels are too low. 


A-31 


5. Are there students who are not in the after-school program who should be? 
o Yes —Please explain: 
) No 
6. Please rate the proportion of your students for whom you think the following statements about the AMP after- 
school program are true. 
All or Hardly 
The AMP after-school program is... almost all mice a ee any 
students students 
a. engaging. 0 Oo Oo 0 0 
b. relevant to their interests. Oo Oo 0 0 0 
C. motivating. Oo Oo Oo 0 0 
d. appropriate to their literacy needs. Oo oO 0 0 0 
€. appropriate to their learning style. Oo Oo 0 0 0 
Use of AMP Software 
7. How comfortable are you with using the Achieving Maximum Potential (AMP) software program? 


o Notcomfortable 

Oo Somewhat comfortable 
o Comfortable 

o Very comfortable 


How frequently do you use the Achieving Maximum Potential (AMP) software for struggling readers in the after- 
school program? 

Never (Continue to 8b) 

Less than once a month (Skip to Q9) 

1-3 times a month (Skip to Q9) 

1-3 times a week (Skip to Q9) 

4-5 times a week (Skip to Q9) 

Multiple times a day (Skip to Q9) 


OoOo0O000 


8b. If you said that you “never” use the AMP software, please indicate why by checking the appropriate 
responses below. (Check all that apply) 


| do not have computers in the classroom where | teach AMP. 

The AMP software is not installed on my computers. 

The computers in the classroom where | teach AMP are not working. 

| do not believe that the AMP software is effective at building students’ literacy skills. 

| do not know how to use the AMP software. 

The reading level of the AMP software is too high for the students who are currently participating. 
The reading level of the AMP software is too low for the students who are currently participating. 
Other (Please specify) 
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Instructional Practices 


9. Inatypical AMP after-school class, how often do you use the following practices or materials with Tier 3 


students to help them increase reading comprehension? 


Less than : : : 
Use of Instructional Practices Never ONES ea eee ee ie ngs 
a month a week a week 
month 
Explicit instruction in the use of one or more of the following 
comprehension strategies: summarizing, questioning, q q q q q 
predicting, text structure, visualization, inferring and 
metacognition 
Establishing the purpose for reading. 0 0 0 Oo Oo 
Monitoring students’ comprehension through questioning. Oo Oo 0 0 0 
Making connections to background knowledge. 0 Oo Oo Oo o 
Making connections between texts. Oo Oo Oo o o 
Synthesizing information within text or across texts. o o o o o 
Using differentiated instruction (i.e. providing different content, 
resources and/or instructional techniques specifically tailored to 0 0 0 0 0 
meet students’ individual educational needs and/or learning styles) 
Use of before, during, and after (BDA) reading strategies for 
comprehension instruction (A student constructed a mental 
framework for reading begun before reading even begins, o o o o o 
strengthened as students interact with the text during the reading, 
and reflected upon after reading.) 
10. Ina typical AMP after-school class, how often do you use the following practices with Tier 3 students to help 
them build their vocabulary knowledge? 
Less than A ; ' 
; ; 1-3times | 1-3times | 4-5 times 
Use of Instructional Practices Never once a Bee cake eee 
month 
Explicit instruction in vocabulary Oo o Oo Oo o 
Modeling the use of word parts o Oo o Oo o 
Review of vocabulary words o o o o o 
Use of before, during, and after (BDA) reading strategies for q q q q q 
vocabulary instruction 
Academic vocabulary for content terms Oo Oo oO Oo Oo 
Word study- word sorts and concepts 0 0 0 0 Oo 
Morphology instruction Oo Oo Oo Oo Oo 
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11. Ina typical AMP after-school class, how often do you use the following practices or materials with Tier 3 


students to help them develop fluency? 


, , reeeeD 1-3timesa | 1-3timesa | 4-5timesa 
Use of Instructional Practices Never once a 
month week week 
month 
Teacher read aloud i) 0 Oo 0 0 
Teacher interactive read aloud 0 0 Oo Oo Oo 
Shared reading (students and teacher take turns in reading) Oo 0 0 0 Oo 
Modeling reading for students 0 0 0 o o 
Explicit instruction in guided oral reading 
Focusing instruction on proper and meaningful phrasing 0 0 0 im) 0 
Students listen to audio books, play aways 0 Oo Oo Oo 0 


12. The gradual release model (Leading students from modeled instruction to shared instruction to guided practice 
and finally students’ independent practice) and explicit instruction in guided reading are only intended to be use 


on an “as needed” basis. 


Duringyour work with students in the AMP after-school program, in a typical classroom, to what extent do 
you feel you are able to meet your Tier 3 students’ individual needs through these instructional practices? 


Extent the technique meets 


Tier 3 Students’ needs 


: : Not 
Instructional Practices and Purposes Using ee Toone ate om To alarge 
extent extent 
extent 

Use of the gradual release of responsibility model q q q q q 
for reading comprehension instruction 

Use of the gradual release of responsibility model q q q q q 
to build vocabulary 

Use of the gradual release of responsibility model q q q q q 
to develop fluency 

Explicit instruction in guided oral reading q q q q q 
to develop fluency 
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Use of Assessment Data 


13. Please indicate the extent to which you use student assessment data for each of the following purposes within 


the AMP after-school program. 


To Some | ToaModerate | ToaLarge 
Use of Assessment Data Not at All Sei Extent Extent 

Differentiating instruction(i.e. providing different 
instructional techniques specifically tailored to meet 

Seki ; Oo 0 0 0 
students’ individual educational needs and/or learning 
styles) 
Identifying skills that need to be taught or retaught. 0 Oo Oo 0 
Monitoring student reading progress. Oo Oo 0 o 
Creating instructional groups (in-class). 0 0 0 Oo 


14. How often do you meet with English language arts teachers at the following grade levels to discuss instruction- 
related issues regarding your work with students in the AMP after-school program? 


Frequency of Meetings with Grade-Level Teachers 
Less than 1-3 1-3 4-5 
Never oncea | timesa | timesa | timesa 
month month week week 
Grade 6 teachers 0 0 | | | 
Grade 7 teachers o 0 0 | 0 
Grade 8 teachers Oo | 0 | 0 
Overall (consider all the teachers that you work with q q q q q 
regardless of the grade level they teach) 


15. How often do you meet with SIXTH-GRADE classroom teachers to discuss implementing each of the following 
instructional methods with students in the AMP after-school program? 


How often discussed with classroom teachers 
Instructional methods and activities for AMP Less than 1-3 1-3 : 
: : 4-5 times 
program Never oncea timesa | timesa aracale 
month month week 
Differentiated instruction(i.e. providing different 
content, resources and/or instructional techniques q q q q q 
specifically tailored to meet students’ individual 
educational needs and/or learning styles) 
Student groupings 0 0 Oo o o 
Use of AMP materials Oo 0 0 Oo Oo 
Using specific AMP and Striving Readers 
instructional techniques for comprehension Oo Oo Oo Oo Oo 
instruction 
Using specific AMP and Striving Readers q q q q q 
instructional techniques for vocabulary instruction 
Using specific AMP and Striving Readers o o o o o 
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How often discussed with classroom teachers 
Instructional methods and activities for AMP Less than 1-3 1-3 45 times 
program Never oncea timesa | timesa Aranek 

month month week 

instructional techniques for fluency instruction 
Discussing specific students’ reading progress. Oo Oo 0 0 0 
Coordinating instruction Oo 0 0 o o 
Using student assessment data for instructional 
planning 


Use of Handheld Computers in the AMPAfter-School Program 


16. a) Do you use handheld computers (Palm Pilots) to teach literacy duringthe AMP class? 
Yes (Skip to Question 17) 
No 


b) If you are NOT using handheld computers during the AMP class, please indicate why you are not using them 
below 


The computers have not been made available for the AMP classes. 

The computers and associated software do not integrate well with the AMP program. 

Some or all of the computers are not working properly. 

Some or all of the necessary software applications have not been installed on the computers. 
| have not received sufficient professional development to feel comfortable using them. 


Because the Striving Readers program provides only 10 computers per classroom, and | do not like to have 
some students use them while others can not. 


| do not feel that they offer sufficient added benefit compared to traditional media (e.g. print, paper and 
pencil) to be worth the trouble. 


Other (please specify): 


Respondent Information 


17. a) At which grade level(s) are you providing targeted intervention support for Tier 2 and 3 students in ELA 
classrooms this year (2009-10)? (Check all that apply): 
060708 


b) For which grade level(s) are you conducting AMP classes this year (2009-10)? (Check all that apply): 
060708 


18. What is the name of your school? [drop down list 
Po HENDRICKS 
BEETHOVEN, | HENSON, 
ees = — = one 


BEETHOVEN 
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GRAY 


19. How many years have you been teaching? [INSERT TEXTBOX] 
20. How many years have you been teaching at this school? [INSERT TEXTBOX] 
21. How many years have you been teaching reading? [INSERT TEXTBOX] 


22. How many years have you been an LIT? [INSERT TEXTBOX] 
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CPS Striving Readers 
Non-ELA Content Area Teachers — Treatment Schools 


The following is a survey designed to gather your feedback on the essential components of the Striving Readers 
program. It will take you approximately 30 minutes to complete. Results will be reported in the aggregate only; we 
will not use your name or identify individual respondents. Your feedback is extremely valuable to the success of this 
program. If you have questions about this survey, please contact Rebecca Swann at rswann@metisassoc.com or 
212-425-8833. 


1. When, if at all, did you start integrating literacy into your content area instruction? 
oO. This year 
oO Last school year 
0 Before Striving Readers began 
0 | do not integrate literacy instruction into my content area. (Skip to Question 5) 


Comprehensive Instruction 


2. Ina typical classroom, how often co you use the following practices to help students increase reading 


comprehension? 

as than | 23 | 23 | 48 

Use of Instructional Practices Never times a | timesa | timesa 
once a 
month week week 
month 

Explicit instruction in the use of one or more of the following 
comprehension strategies: summarizing, questioning, 
predicting, text structure (the organizational arrangements q q q q q 
used to present information), visualization, inferring and 
metacognition (students select appropriate comprehension 
strategies) 
Establishing the purpose for reading. 0 0 0 Oo Oo 
Monitoring students’ comprehension through questioning. 0 0 0 0 Oo 
Making connections to background knowledge. 0 0 0 0 Oo 
Making connections between texts. oO 0 0 Oo o 
Synthesizing information within text or across texts. Oo 0 0 0 0 
Using differentiated instruction (i.e. providing different content 
and/or instructional techniques and resources specifically tailored to 0 0 0 o o 
meet students’ individual educational needs and/or learning styles) 
Use of before, during, and after (BDA) reading strategies for 
comprehension instruction (A student constructed mental 
framework for reading begun before reading even begins, 0 0 0 Oo Oo 
strengthened as students interact with the text during the 
reading, and reflected upon after reading.) 
Using Pairing for Partner Reading & Content Too (PRC2) for q q q q q 
comprehension instruction 
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3. How often do you use the following practices to help students build their vocabulary knowledge? 


Use of Instructional Practices 


Explicit instruction in vocabulary 


Never 


Less 
than 
oncea 
month 


o 


1-3 
times 
a 
month 


Oo 


1-3 
times a 
week 


Q) 


4-5 
times a 
week 


Q) 


Modeling the use of word parts 


Review of vocabulary words 


Use of vocabulary notebooks 


PRC2 for vocabulary development. 


vocabulary instruction 


Use of before, during, and after (BDA) reading strategies for 


Academic Vocabulary for content terms (e.g., Marzano) 


Morphology instruction (e.g,, Shane Templeton) 


Word study sorts and concepts (e.g,, Donald Bear) 


Words Their Way (e.g,, Donald Bear & Shane Templeton) 


OQ);QO;/O/;0); OQ ;O;0/;0;0/0 


OQ);O;0O/;0) O,;0;0;0/0 


Q);QO;/0O/;0) 0 ,;0;0;0/0 


OQ;)O;/O/;0) 0 ,;0;0;0/0 


OQ)/O;/O/;0); 0 ,;0;0;0/0 


4. Ina typical classroom, how often do you use the following techniques to help students develop better 


reading strategies and skills? 


Techniques 


Never! Not 
Familiar 


Less than 
once a month 


1-3 times 
a month 


1-3 times 
a week 


4-5 times a 
week 


Everybody Reads To (ERT) 0 


Oo 


Oo 


qo 


Oo 


Exclusion Brainstorming 


List-Group-Label 


Predict-Locate-Add-Note (PLAN) 


ReQuest 


Interactive Notation System for Effective Reading and 
Thinking (INSERT) 


ABC Graffiti 


Guided Reading and Summarizing Procedure 
(GRASP) 


OQ}; OQ /;O; 0 ;O;O;O;0 


KWL 


OQ}; Q/;O; 0 ;O;O;O/;0 


OQ}; OQ /;O; 0 ;O;O;O;0 


OQ}; OQ /;O; 0 ;O;O;O;0 


OQ}; Q/;O; 0 ;O;O;O;0 
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5. Please indicate how often (if at all) you discussed each of the following topics with any literacy experts 


during the current school year. 


(Literacy experts could include any of the following: school staff [Literacy Team members, Literacy 


Coaches]; district staff [the LIT, the Striving Readers Coordinator]; NLU consultants [Donna Ogle, Debbie 
Gurvitz]; other consultants [Donald Bear, Shane Templeton, Doug Fisher]; speakers at SR conferences; 


course instructors; etc.) 


Less than 1-3 1-3 2 
: : ; 3 : : : 4-5 times 
Discussion Topics with a Literacy Expert Never oncea timesa | timesa 
a week 
month month week 
a. Differentiated instruction (i.e. providing different content and/or instructional 
techniques and resources specifically tailored to meet students’ individual 0 0 Oo o o 
educational needs and/or learning styles) f 
b. Student groupings 0 0 0 0 0 
c. Use of Striving Readers text sets and text sets teacher guides. 
[These are sets developed and distributed by the Striving Readers program o o o oO o 
that include collections of short books centered around specific content area 
themes, written at a variety of reading levels so students can access the 
books independently. ] 
d. Use of technology resources: desktop computers, handhelds- palms, LCD 
projector, etc. 
e. Use of PRC2 instructional framework 
f. Using specific Striving Readers instructional techniques for comprehension 
instruction 
g. Using specific Striving Readers instructional techniques for vocabulary 
instruction 
h. Specific students’ reading progress. 0 im) 0 0 0 
i. Using student assessment data for instructional planning 0 Oo 0 o o 
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School-wide Intervention Materials 


Use of Striving Readers Text Sets and Text Sets Teacher Guides 


Striving Readers text sets are defined hereas collections of short books centered around specific content area 
themes, written at a variety of reading levels so students can access the books independently. These text sets were 
developed and distributed by the Striving Readers program 


6. a) Do you use Striving Readers text sets to teach literacy through your content area? 


Yes (SKIP to 
No ( 


b) Ifyou are NOT yet using Striving Readers text sets, please indicate why you are not using them below 
(Check all that apply) (After this item, respondents will proceedto aaa 


They have not been made available to me. 


Other (specify): 


The content is not relevant/interesting to my students. 
The Striving Readers text sets are not sufficiently aligned to the curriculum of my subject area. 

The range of reading levels covered by the Striving Readers texts sets is too high for my students. 
The range of reading levels covered by the Striving Readers texts sets is too low for my students. 
| did not receive Striving Readers text sets for the topics that we are covering in my class. 
| have Striving Readers texts sets, but not the text sets teacher guides. 


11) 


7. Please indicate the proportion of your students for whom the following statements are true. 


All or 
almost all 
students 


Most 
students 


A few 
students 


Hardly 
any 
students 


The range of reading levels covered by the Striving 
Readers text sets is too high for these students. 


o 


qo 


o 


Oo 


The range of reading levels covered by the Striving 
Readers text sets is too low for these students. 


The range of reading levels covered by the Striving 
Readers text sets is appropriate for these students. 


Striving Readers text sets are relevant to their 
interests. 


Striving Readers text sets are appropriate to their 
learning style. 


Striving Readers text sets motivate students to learn 
more about a topic. 


Striving Readers text sets are appropriate to their 
literacy needs. 


OQ);o;a;a;0;0 


OQ;o;a0;a;0;0 


Q);oOo;a;a;0;0 


OQ;oOo;0;Q0;0;0 


8. Please describe your use of the Striving Readers text sets to teach comprehension strategies and to 
provide access to leveled content reading materials in your typical content area classroom. Use the 
table below to indicate the units with which they are used and the duration of those units. 
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Identify those units for which Identify text set(s) used Typical duration of 
you use SR text sets for this unit 
1) __days/___ weeks 
2) ___days/___ weeks 
=) ___days/___ weeks 
A) ___days/___ weeks 
5) 


__days/___ weeks 


9. Rate how comfortable you are with using the Striving Readers text sets to support student learning in 


language arts. 


. 2 3 4 : 

Not at all Very 

Comfortable Comfortable 
Oo a) Oo Oo i) 
10. In your opinion, how well are the Striving Readers text sets aligned to the curriculum of your subject 
area? 
1 2 3 4 5 
Poorly Adequately Very well 

Aligned Aligned aligned 

Oo Oo Oo oO Oo 
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Professional Development in Literacy Instructional Practices 


11. For each of the following literacy teaching practices, indicate: 


a Did you receive professional development through the Striving Readers program during the current school year? 


e Yes 
@ No (Skip to Q 14) 
If so: 


b. Please indicate whether you are using this practice as part of your content area instruction, and 
c. Rate your comfort implementing each teaching practice within your content area instruction. 


Literacy based teaching practices 


é 
i 


7 


IF YES: 


Cc) Rate your confort level 


Not at all 


2 3 4 


Building academic vocabulary 


Using classroom libraries 


Creating literacy-rich classroom environments 


Differentiating instruction 


Explicit vocabulary instruction 


Incorporating text sets and teacher guides in your instruction 


Increasing student motivation 


Supporting students’ self-directed learning 


Using before, during, and after reading strategies 


Using formal assessments to guide instruction 


Using informal assessments to guide instruction 


Using handheld computers (Palm Pilots) 


Using literacy-based software 


Using the PRC2 model 


Using the whole-part-whole classroom instruction model 


OQ/O;)QO;)QO;O/;/0/0/0/0/0)/0)0;0;0/0 


OQ/O;)QO;)O;O/;/0/0/0/0/0)/0)0;0;0/0 


OQ/O,)QO;)QO;O/;/0/;/0/0/0/0)/0)0;0;0/0 


OQ/O;)QO;)O;O/0/;/0/0/0/0;/0)0;0;/0/0 


s/o slalala|a/oolalalalalaag 
) 


OQ/O;)QO;)O;/O/0/0/0/0/0;/0)0;0;/0/0 
OQ/O;)Q;)O;O/;/0/;/0/0/0/0;/0)0;0;0/0 
OQ/O;)QO;)O;/O/0/;/0/0/0/0)/0)0;0;/0/0 


OQ)/O;)QO;)O;/0O/;/0/0/0/0/;0;0;0);0/0 
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12. Please check the techniques and strategies in the list below for which you would like to receive more 
training. (Check all that apply) 


Academic Vocabulary for content terms (e.g., Marzano) 
Morphology instruction (e.g,, Shane Templeton) 

Word study sorts and concepts (e.g,, Donald Bear) 
Words Their Way 

KWL 

Using PRC2 for fluency instruction. 

Using PRC2 for comprehension instruction. 

Using PRC2 for vocabulary development. 

Everybody Reads To (ERT) 

Exclusion Brainstorming 

List-Group-Label 

Predict-Locate-Add-Note (PLAN) 

ReQuest 

Interactive Notation System for Effective Reading and Thinking (INSERT) 
Read Aloud/Think Aloud 

ABC Graffiti 

Guided Reading and Summarizing Procedure (GRASP) 
Teaching summarizing as a comprehension strategy 
Teaching questioning as a comprehension strategy 
Teaching predicting as a comprehension strategy 
Teaching text structure as a comprehension strategy 
Teaching visualization as a comprehension strategy 
Teaching inferring as a comprehension strategy 
Teaching metacognition as a comprehension strategy 


OUCUOOOODOOOCOOCOOOODOOOUVCOUVCOD 


Respondent Information 


13. What is your position? O General Education Teacher OBilingual/ELL teacher MSpecial education 


teacher 0 LIT © Reading Specialist Other (Please Specify: ) 


14. What subject(s) do you teach? (Check all that apply) 

C1 All subjects 

© Literacy/Reading/English language arts 

Mathematics 

Science 

Social studies 

Other: (Please Specify: ) 


15. At which grade level(s) are you teaching this year (2009-10)? (Check all that apply): 


0k010203040506070809 010 011 012 
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16. What is the name of your school? [drop down list 


GRAY 


17. How many years have you been teaching? [INSERT TEXTBOX] 


18. How many years have you been teaching at this school? [INSERT TEXTBOX] 
19. How many years have you been teaching math? [INSERT TEXTBOX] 

20. How many years have you been teaching science? [INSERT TEXTBOX] 

21. How many years have you been teaching social studies? [INSERT TEXTBOX] 


22. How many years have you been teaching English language arts? [INSERT TEXTBOX] 


Teachers will be reimbursed by CPS-Striving Readers for their time to complete this survey. In order to be reimbursed we need 
you to identify yourself so that we can verify that you completed the survey. If you would like to be reimbursed, please provide 
your name and email address below, and be sure that you identified your school in the previous item. Your survey responses 
will still remain strictly confidential and will never be reported in any form that would allow anyone to connect your responses with 
your name. Providing this information is optional. 


Your Name: Email: 


Thank you for completing this survey! 
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Chicago Public Schools (CPS) Striving Readers 
Spring 2010 Literacy Improvement Survey for Teachers - CONTROL SCHOOLS 


The following is a survey designed to gather your feedback on the essential components of your school’s literacy program. 
Survey results will be reported in the aggregate only. We will not use your name or identify individual respondents. Your 
feedback is extremely valuable to the success of this program. If you have questions about this survey, please contact Rebecca 
Swann at rswann@metisassoc.com or 212-425-8833. 


49. What is your primary role or teaching assignment? 


(Select the single best option. 


General education teacher (self-contained classroom teacher) (Continue with LIS) 
English language arts teacher 
Teach English language arts and other academic subject 0 
Teach other academic subjects but notEnglish language arts 

Bilingual/ELL teacher 
Special education teacher 
Reading specialist 
Other (please specify): 


Does this role include teaching of English language arts? 
e §6Yes (Continue with LIS) 


e No (Jump to “Thank you for completing this survey!”) 
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Comprehensive Instruction 


1. Ina typical classroom, how often do you use the following practices to help students increase reading comprehension? 


Less 


1-3 


Not than | times si 
Use of Instructional Practices on Never timesa | timesa 
Familiar once a a week week 
month | month 
Explicit instruction in the use of any one or more of the following 
comprehension strategies: summarizing, questioning, 
predicting, text structurevisualization, inferring and 7 = 0 2 0 7 
metacognition 
Establishing the purpose for reading. 0 0 0 0 o o 
Monitoring students’ comprehension through questioning. 0 0 i) i) o Oo 
Making connections to background knowledge. Oo 0 0 0 0 0 
Making connections between texts. Oo 0 0 0 o o 
Synthesizing information within text or across texts. o 0 0 0 o oO 
Using differentiated instruction (i.e. providing different content, 
resources and/or instructional techniques specifically tailored to 
meet students’ individual educational needs and/or learning ey 4 a = a a 
styles) 
Use of before, during, and after (BDA) reading strategies for 
comprehension instruction (A student constructed mental 
framework for reading begun before reading even begins, 0 0 im) 0 0 0 
strengthened as students interact with the text during the reading, 
and reflected upon after reading.) 
Using partner reading strategies for comprehension instruction Oo =) =) 0 o oO 
2. Ina typical classroom, how often do you use the following practices to help students build their vocabulary knowledge? 
Not Less 1-3 
; Familiar than | times | 2° a 
Use of Instructional Practices Never ence FA times a | timesa 
month | month wee en 
Explicit instruction in vocabulary o i) 0 0 0 0 
Modeling the use of word parts o 0 Oo Oo o o 
Review of vocabulary words o 0 o o o o 
Use of vocabulary notebooks o 0 o o o o 
Use of partner reading strategies for vocabulary development. o 0 Oo o oO o 
Use of before, during, and after (BDA) reading strategies for o q q q q qo 
vocabulary instruction 
Words Their Way o 0 Oo 0 0 im) 


Not Less 1-3 
: : Familiar than | times | 2° sa 
Use of Instructional Practices Never ones a times a | timesa 
month | month wees wes 
Academic Vocabulary for content terms (e.g., Marzano) im) 0 Oo 0 0 0 
Word study sorts and concepts (e.g,, Donald Bear) o o 0 0 0 o 
Morphology instruction (e.g,, Shane Templeton) 0 0 Oo 0 0 o 
3. _Ina typical classroom, how often do you use the following practices to help students develop fluency? 
Not ite fies is seis 
Use of Instructional Practices ¥ Never timesa | timesa 
Familiar once a a ech week 
month | month 
Teacher read aloud i) 0 im) 0 o 0 
Teacher interactive read aloud 0 im) 0 0 o 0 
Shared reading (students and teacher take turns in reading) Oo 0 i) 0 o 0 
Modeling reading for students i) im) 0 0 o 0 
Explicit instruction in guided oral reading i) im) i) 0 o 0 
Students listen to audio books, play aways i) im) i) 0 o 0 


4. Ina typical classroom, how often do you use the following techniques to help students develop better reading strategies and 


skills? 
: - Less than once | 1-3timesa | 1-3timesa | 4-5timesa 
Techniques Not Familiar Never 
a month month week week 
Everybody Reads To (ERT) 0 0 im) 0 0 0 
Exclusion Brainstorming 0 Oo 0 0 0 0 
List-Group-Label i) i) im) 0 0 0 
Predict-Locate-Add-Note (PLAN) i) 0 im) 0 0 i) 
ReQuest 0 0 0 o o o 
Interactive Notation System for Effective Reading and 
Thinking (INSERT) 2 z eo 2 2. z 
ABC Graffiti 0 i) im) i) 0 0 
Guided Reading and Summarizing Procedure (GRASP) 0 0 0 0 0 i) 
KWL 0 0 0 i) Oo 0 
5. _ Ina typical classroom, how often do you use the following grouping structures? 
P Eessan 1-3 times a 1-3timesa | 4-5timesa Muniple 
Grouping Structures Never oncea times a 
month week week 
month day 

Whole class/Large group o o oO o 0 i) 

Individual Work oO o o Oo o Oo 

Small groups or Pairs o o o o o Oo 
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6. Considering your own instruction (not that of literacy support staff or other instructors in your classroom), in a typical 
classroom, how often do you apply differentiated instruction (providing different content, resources and/or instructional 
techniques specifically tailored to meet students’ individual educational needs and/or learning styles) in your classroom? 


Almost 
Never Rarely Occasionally en nate mos! of the eee 
time time lesson or 
activity 
im) Oo 0 im) 0 im) 
Purposeful Assessment 
7. _ Indicate how you use the data from the following assessments. (Please check all that apply.) 
' “ : . : Progress Assess 
Assessments Not Using Screening Diagnostic Benchmarking Monitoring Guicanics 
Reading Benchmark Assessment im) 0 0 0 0 o 
Illinois Standards Achievement qo qo q qo qo qo 
Test 
Basic Reading Inventory (BRI) 0 0 im) Oo 0 o 
Informal assessments 0 0 0 0 Oo o 
Fluency Snapshots im) 0 0 0 0 0 
Spelling Inventories im) 0 0 i) 0 0 
Other: 0 0 0 Oo 0 0 
Other: 0 im) im) Oo oO 0 
Other: im) 0 0 i) 0 0 
Data-Driven Instruction 
8. Please indicate the extent to which you use student assessment data for each of the following purposes. 
Use of Data Not at All To Some extent Tipe Monee To a Large Extent 
Extent 
Placing students in intervention programs. 0 0 0 Oo 
Differentiating instruction (i.e. providing 
different content, resources and/or 
instructional techniques and materials 5 q q 5 
specifically tailored to meet students 
individual educational needs and/or learning 
styles). 
Identifying skills that need to be taught or 
retaught. 
Monitoring student reading progress. 0 im) Oo 0 
Creating instructional groups (in-class). i) im) Oo 0 


Grade-Level Teams 


9. Do you currently have grade-level teams at your school? (Grade-Level Teams are teams consisting of staff across subject 
areas from the same grade, or in grade level “bands”.) 


o Yes 


0 No (If no, skip to Question 12) 
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10. Overall, rate the grade-level team’s ability to use classroom assessment data in the following ways. 


Use of Data 


Poor 


Good 


Excellent 


Not Sure 


Address the literacy needs of all students. 


Oo 


mi) 


mi) 


og 


Address the needs of struggling readers. 


Formalize lesson plans. 


Identify students who are eligible for targeted 
interventions. 


Identify strengths. 


Identify teaching and learning strategies. 


Improve classroom practice. 


Q;)O/;0; OQ ;oO;a 


QOQ)O);O);) OQ ;O/;O 


Q)O)O);) QQ ;O/;0 


O;/Q/;0O) Q ;O/;O 


Literacy Teams 


11. Do you currently have a literacy team in place at your school? (A literacy team is a team that focuses on literacy issues 


across grade levels.) 
o Yes 
o No (If no, skip to Question 14) 


12. Overall, rate the quality of the literacy team’s performance in the following areas. 


Performance Areas Poor Fair Good Excellent Not Sure 
oe 2/9 | o [| o | 3p 
Addressing the needs of all students. im) 0 im) 0 0 
Addressing the needs of struggling readers. 0 0 im) 0 0 
Addressing the needs of grade-level teams. im) 0 im) 0 0 
Improving literacy instruction at your school. im) i) im) im) 0 
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School-wide Intervention Materials 


13. For each of the materials listed below, 
Indicate how frequently you currently use the materials to teach literacy in your typical classroom. 


For those that you are using, rate how comfortable you are with using these materials to support student learning in language arts. 


ie b) IF USING: 
Please rate your confort level 

Materials NA Not Lessthan | 1to3 1to3 A4to5 1 2 3 A: 5 

(Do Not } Currently oncea times a times a times a Not at all Very 

Have) Using month month week week Comfortable Comfortable 
Listening centers (Classroom CD 
sa aS aaa o o o fm) o o o Oo o oO fm) 
headphones) 
brehas Giese o o o oO o o o Oo Oo Oo oO 
Classroom library o o o o o o o 0 o o o 
Vocabulary notebooks oO Oo oO o o Oo | Oo | Oo Oo 
Reading response notebooks o o o o o o o o oO 0 Oo 
Schoo library og o oO og Oo Oo | Oo | Oo Oo 
Reading Anthologies og o Oo og Oo oO | o | Oo Oo 
Reading Basals oO o Oo o Oo Oo | Oo | Oo Oo 
mea ae bd ale oO Oo o Oo oO o o Oo o o o 
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14. For each of the materials listed across the top of the chart below, please indicate which literacy instructional goals are supported by your use of that material in your 


classroom. (check all that apply.) 
Listening Classroom Readingbasals School 
Instructional goals that each material is centers Media centers library L Library 
used to support: J J J 
O o a = a a = 


Materials: 


e 
eo 


Vocabulary Reading 
notebooks anthologies 
1 
Oo 


Vocabulary Development 
Grammer 


|Gammer 


To develop students’ self-directed learning 
To supplement students’ textbook reading 


Teaching students to identify and use text 
structure 


Teaching students to identify and use the 


organizational features of expository writing 
To activate students’ prior knowledge 


o 
o 


Oo 


A-52 


15. For each of the materials listed below, please indicate whether the following grouping strategies or differentiated instruction 
are supported by your use of these materials in your classroom. (Check all that apply) 


Instructional Groupings and Differentiation 
Materials iA Whole Class! | Small Group! | Individual ae Hea d 
Large Group Pairs Work inetiticton 
Listening centers o qo Oo qo go 
Media centers o o Oo q go 
Text sets 0 Oo 0 Oo 0 
Classroom library 0 im) Oo q qo 
Vocabulary notebooks 0 0 0 fm) Oo 
Textbooks 0 Oo im) oO Oo 
Reading response notebooks 0 0 0 qo o 
School Library 0 qo Oo q qo 
Reading anthologies 0 0 0 Oo 0 
Readingbasals o im) Oo qo qo 
Other informational texts (other than text sets) 0 0 0 0 0 


Use of Cl 


assroom Computers for Literacy Instruction 


16. a) Do you use classroom computers (desktop, laptop, or handheld computers)to teach literacy? 


Yes (Skip to Question 18.) 
No 


b) If you are NOT yet usingclassroom computers to teach literacy, please indicate why you are not using them below 
(Check all that apply) and then SKIP to Question 22: 


We do not have computers in our classrooms. 

Some or all of the computers are not working properly. 

Some or all of the necessary software applications have not been installed on the computers. 
| have not received sufficient professional development to feel comfortable using them. 


We do not have enough computers for every student and | do not like to have some students use them while others 
can not. 


| do not feel that they offer sufficient added benefit compared to traditional media (e.g. print, paper and pencil) to be 
worth the trouble. 


Other (please specify): 


17. In atypical classroom, how often do your students use classroom computersduring literacy instruction? 


oO 


oO 
oO 
oO 


Less than once a month 
1-3 times a month 

1-3 times a week 

4-5 times a week 
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18. 


Rate how comfortable you are with using the classroom computersto support your literacy instruction 


Not at all Comfortable Very Comfortable 


1 2 3 4 5 


o o o oO qo 


19. 


20. 


Which specific academic foci or instructional objectives do you support with the use of the classroom computers? (Check all 
that apply) 

1 Fluency 

Vocabulary development 


Developing students’ reading comprehension strategies 
1 Writing skills 

1 Word parts 

1 Word recognition 


1 Spelling 
Grammar 


Locating information 
Evaluating information 


Synthesizing information 
1 Organizing information 
To develop students’ self-directed learning 


1 Teaching students to identify and use the organizational features of expository writing 
To activate students’ prior knowledge 


Which of the following instructional activities and practices do you support with the use of the classroom computers? (Check 
all that apply) 


[11 Whole class/ Large group 

Small group/ Pairs 

1 Individual Work 

1 Monitoring distribution and completion of assignments 
Assessing students’ literacy skills 

1 Monitoring students’ progress 


Differentiating instruction (i.e. providing different content, resources and/or instructional techniques specifically tailored 
to meet students’ individual educational needs and/or learning styles) for struggling readers 


Differentiating instruction for English language learners/special education students 
Guided reading 

Partner reading 

O Individual reading 

Book club discussions 
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Classroom Library 


21. 


22. 


23. 


24. 


Please check the ways that you use your classroom libraries. (Check all that apply) 
1 For content area instruction 

1 For independent reading 

For small group instruction 

11 For read alouds 

| do not have a classroom library [SKIP to Q28] 


Do you use interest inventories to help students self select reading material? 
o Yes 
o No 


Do you use interest inventories to guide your purchases for the classroom library? 
o Yes 
o No 


Please indicate how true each of the following statements are about the organization of books in your classroom library. 


My classroom library... True True True 


Not At All Slightly Somewhat Very True 


...IS easily accessible to students. 0 0 im) 0 


...IS well organized and in good shape. 


..has a checkout system in place. 


... includes a variety of reading materials that are 
appropriate for readers of differing abilities. 


.. includes a variety of texts that appeal to readers with 
differing interests. 


..has reading materials grouped by genre. 


...has reading materials clearly labeled. 


..has both nonfiction and fiction books. 


QO) O; 0 ;o;0 


QO) O; O;o;a 


QO) O; O;oO;a 


QO) O; O;oO;a 


25. To what extent are you able to consider students’ needs and reading abilities when ordering books and other reading 


material for your classroom library? 
o Notatall 

To asmall extent 

To a moderate extent 

To a large extent 

Don’t know 


oo0o°0 


26. To what extent are you able to consider students’ interests and motivation when ordering books and other reading 


material for your classroom library? 
o- Notatall 

To a small extent 

To a moderate extent 

To a large extent 

Don't know 


oo0°0 


School Library 


27. Ina typical classroom, how often do you take your class to the library? 


oOo Never 
oO Rarely (less than once a month) 
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28. 


29. 


30. 


31. 


32. 


O 
O 
O 


Sometimes (at least once a month) 
Often (at least once a week) 
Almost daily or daily 


To what extent do the library resources support your school’s literacy goals? 


Oo 


ooo0o°o 


Not at all 

Toa small extent 

To a moderate extent 
To a large extent 
Don’t know 


Do you have a school librarian? 


Yes 
No [Skip to question 34] 


How does the librarian work with you? (Check all that apply.) 


oO 


O 
O 
O 


The librarian does not work with me. 

The librarian provides resources for class projects. 

The librarian and | collaborate on how to supplement lessons with library resources. 
Other (please specify): 


To what extent does the librarian consult with classroom teachers to order reading materials that are grade level and 


content appropriate? 

o- Notatall 

oO Toasmall extent 

oO Toamoderate extent 
o Toalarge extent 

o Don't know 


How does the librarian work with your students? (Check all that apply.) 


Oo 


ooo0o°o 


ooo0o°o 


Does not work with my students. 

Works with students on research skills. 

Directs students to resources tied to curriculum. 

Conducts read-alouds. 

Provides students with information about extracurricular academic activities (e.g., spelling bee, writing competitions, 
events). 

Assists students with class projects. 

Teaches students how to navigate Internet resources. 

Guides struggling readers to summer programs. 

Other (please specify): 


Collaboration with Literacy Support Staff 


33. Does your school have a literacy enrichment specialist such as a Reading Specialist or Literacy Coach? 


Yes 
Please indicate this person’s title: 
No (Skip to Question 39) 


34. How often do you meet or collaborate with the literacy enrichment specialist in the following settings? 


. Less than 1-3 times a 1-3 times a 4-5 times a 
Meeting Structures Never 
once a month month week week 
Scheduled one-on-one meetings o 0 im) 0 0 
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Impromptu one-on-one meetings (during lunch, 4 5 q q q 
prep periods, before/after school, etc.) 

Grade-level team meetings o o o Oo o 
Literacy team meetings o o Oo 0 Oo 


35. To what extent has your collaboration with the literacy enrichment specialist facilitated your efforts to use the following 
methods to support struggling readers in your class? 


ODid not collaborate (skip to Question 37) 


Extent to which collaboration with the literacy 
enrichment specialistfacilitated use of methods 


Instructional methods Toa Toa 
To a large 
Not at all small moderate 
extent 
extent extent 

j. Differentiating instruction o | Oo | 
k. Scaffolding of instruction Oo | Oo | 
l. Student groupings go oO Oo | 
m. Using the media center o | Oo | 
n. Using listening centers og | Oo | 
0. Using handheld computers Oo Oo o 0 
p. Using assessment data to monitor qo q qo q 

student progress 
q. Using student assessment data for qo q qo q 

instructional planning 


36. To what extent has your collaboration with the literacy enrichment specialist facilitated your ability to provide effective 
instruction in the following areas for struggling readers? 


ODid not collaborate (skip to Question 38) 


Extent to which collaboration with the literacy 
enrichment specialistfacilitated effective instruction 
Academic areas Toa 
Toasmall To a large 
Not at all moderate 
extent extent 
extent 
a. Comprehension i) i) i) 0 
b. Fluency 0 i) 0 0 
c. Vocabulary 0 0 0 0 
d. Writing skills 0 0 i) 0 
e. Word parts 0 i) 0 0 
f. Word recognition im) 0 i) 0 
g. Spelling i) 0 0 0 
h. Readingj/literacy in q q q q 
content areas 
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37. Overall, how effective has the literacy enrichment specialist been in improving the reading skills of struggling readers in your 
classroom? 
o Notatall effective 

Minimally effective 

Somewhat effective 

Effective 

Very effective 


Oo0o0°0 
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Professional Development 


38. For each of the following topics, indicate: 


=~ Whether you received professional development addressing this topic during the current year 
= If so, rate the impact that professional development you received has had on your comfort with each teaching practice. 


If YES, what impact did the professional development have on your comfort 


Teaching practices eros with each teaching practice? 
No Yes No Impact Slight Impact Moderate Impact Major Impact 

Building academic vocabulary o o o o oO Oo 
Classroom libraries 0 0 0 0 0 o 
Creating literacy-rich classroom environments o o o o o oO 
Differentiating instruction o o o o Oo Oo 
Explicit vocabulary instruction o o o o og Oo 
Increasing student motivation o o o o o Oo 
Supporting students’ self-directed learning o o o o Oo Oo 
Using before, during, and after reading strategies and techniques 0 o o o oO Oo 
Using student assessments to guide and inform instruction o o o o o Oo 
Using classroom computers o o o o Oo Oo 
Using literacy-based software o o o o Oo oO 
Using partner reading o o o o o oO 
Using the whole-part-whole classroom instruction model o o o o o Oo 
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Struggling Readers: Extended Day (Afterschool) Intervention 


39. Does your school currently offer after or before school programming specifically targeting struggling readers? 
Oo Yes 
Oo No (Ifno, skip to Question 43.) 


40. How many of your current students are involved in the after or before school program? 
None (If none, skip to Question 43.) 

1 to3 

4to6 

7to9 

10 or more 


oo0o00 


41. Overall, how effective has the after or before school component been in improving the literacy abilities of 
struggling readers? 
Oo Notat all effective 


Oo Minimally effective 

oO Somewhat effective 

o Effective 

o. Very effective 

oO Don’t Know 
About You 


42. In addition to English language arts, what other subject areas do you teach? 
Mathematics ( 
Science ( 
Social studies ( 


Other: (Please Specify: ) 


About You [Alll types of respondents] 


43. What is the name of your school? 
[drop down list] 
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44. At which grade level(s) are you teaching reading/English language arts this year (2009-10)? (Check all that 
apply): 


0k010203040506070809 010 011 012 


45. In which of the following settings do you teach literacy? (Check all that apply) 
oO Self-contained 

Subject-Area specialist 

Departmentalized 

Double block 

Other (Please specify): 


oo0o0°0 


46. How many years have you been teaching? [INSERT TEXTBOX] 
47. How many years have you been teaching at this school? [INSERT TEXTBOX] 


48. How many years have you been teaching reading? [INSERT TEXTBOX] 


Teachers will be reimbursed by CPS-Striving Readers for their time to complete this survey. In order to be reimbursed we need 
you to identify yourself so that we can verify that you completed the survey. If you would like to be reimbursed, please provide 
your name and email address below, and be sure that you identified your school in the previous item. Your survey responses 
will still remain strictly confidential and will never be reported in any form that would allow anyone to connect your responses with 
your name. Providing this information is optional. 


Your Name: Email: 


Thank you for completing this survey! 
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CPS Striving Readers 
Non-ELA Content Area Teachers — CONTROL SCHOOLS 


The following is a survey designed to gather your feedback on the essential components of your school's literacy 
program. It will take you approximately 30 minutes to complete. Results will be reported in the aggregate only; we 
will not use your name or identify individual respondents. Your feedback is extremely valuable to the success of this 
program. If you have questions about this survey, please contact Rebecca Swann at rswann@metisassoc.com or 


212-425-8833. 


1. When, if at all, did you start integrating literacy into your content area instruction? 


oO. This year 
oO Last school year 
O Two or more years ago 


0 | do not integrate literacy instruction into my content area. (Skip to Question 5) 


Comprehensive Instruction 


2. Ina typical classroom, how often do you use the following practices to help students increase reading 


comprehension? 


Use of Instructional Practices 


Not 
Familiar 


Never 


Less 
than 
oncea 
month 


1-3 
times 
a 
month 


1-3 
times 
a 
week 


4-5 
times 
a 
week 


Explicit instruction in the use of one or more of the 
following comprehension strategies: summarizing, 
questioning, predicting, text structure (the 
organizational arrangements used to present information), 
visualization, inferring and metacognition (students 
select appropriate comprehension strategies) 


Establishing the purpose for reading. 


Monitoring students’ comprehension through questioning. 


Making connections to background knowledge. 


Making connections between texts. 


Synthesizing information within text or across texts. 


QO)O;O);0/;0 


O;O/;O);0O/;0 


QO)O/;O);0O/;0 


QO)O/;O);0O/;0 


OQ)O/;O);0O/;0 


QOQ)O/;O);0O/;0 


Using differentiated instruction (i.e. providing different 
content and/or instructional techniques and resources 
specifically tailored to meet students’ individual educational 
needs and/or learning styles) 


Use of before, during, and after (BDA) reading strategies 
for comprehension instruction (A student constructed 
mental framework for reading begun before reading even 
begins, strengthened as students interact with the text 
during the reading, and reflected upon after reading.) 


Using partner reading to enhance comprehension 
instruction 
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3. How often do you use the following practices to help students build their vocabulary knowledge? 


Not Less 1-3 1-3 4-5 

Use of Instructional Practices Families Never Unat { silmese | Stimess | anes 

oncea a a a 

month | month | week | week 
Explicit instruction in vocabulary 0 0 0 i) Oo 0 
Modeling the use of word parts 0 0 0 i) Oo 0 
Review of vocabulary words Oo Oo 0 Oo 0 Oo 
Use of vocabulary notebooks Oo Oo Oo 0 Oo Oo 
Use of partner reading to enhance vocabulary q q q q q q 
development. 
Use of before, during, and after (BDA) reading strategies q q q q q q 
for vocabulary instruction 
Academic Vocabulary for content terms (e.g., Marzano) 0 Oo Oo 0 Oo Oo 
Morphology instruction (e.g,, Shane Templeton) 0 0 0 0 0 0 
Word study sorts and concepts (e.g,, Donald Bear) 0 0 0 0 0 0 
Words Their Way (e.g,, Donald Bear & Shane q q q q q q 


Templeton) 


4. Ina typical classroom, how often do you use the following techniques to help students develop better 


reading strategies and skills? 


Techniques 


Not Familiar 


Never 


Less than 
once a month 


1-3 times 
a month 


1-3 times 
a week 


4-5 times a 
week 


Everybody Reads To (ERT) 


Q) 


o 


o 


o 


Oo 


Exclusion Brainstorming 


List-Group-Label 


Predict-Locate-Add-Note (PLAN) 


ReQuest 


Interactive Notation System for Effective Reading and 
Thinking (INSERT) 


ABC Graffiti 


Guided Reading and Summarizing Procedure 


(GRASP) 


KWL 


OQ; OQ |O; OU joO;o;oO;O/;0 


QO; Q ;O; 0 ;Oo;o;ao;0 


QO; Q |;O; 0 ;o;o;o;0 


OQ; OQ |O; 0 ;Oo;o;oO;0 


QO; Q |O; 0 ;oO;oO;aO;0 


OQ; Q |;O; O&O ;o;o;ao;0 
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5. Please indicate how often (if at all) you discussed each of the following topics with any literacy experts 


during the current school year. 


(Literacy experts could include any of the following: school staff [Literacy Team members, Literacy 
Coaches]; CPS district staff; academicor other consultants; speakers at literacy conferences; course 


instructors; etc.) 


Less than 1-3 1-3 : 
; : , : ; : . 4-5 times 
Discussion Topics with a Literacy Expert Never oncea timesa | timesa 
a week 
month month week 
a. Differentiated instruction (i.e. providing different content and/or instructional 
techniques and resources specifically tailored to meet students’ individual Oo Oo Oo Oo Oo 
educational needs and/or learning styles) f 
b. Student groupings 0 0 0 0 0 
Cc. Use of text sets. 
i) i) 0 0 0 
[These are sets of texts that include collections of short books of different 
reading levels, centered around specific content area themes. ] 
d. Use of technology resources: desktop computers, classroom 4 5 4 4 4 
computers/laptops, LCD projector, etc. 
e. Use of the partner reading instructional technique i) 0 i) 0 0 
f. Using specific instructional techniques for comprehension instruction Oo Oo 0 Oo Oo 
g. Using specific instructional techniques for vocabulary instruction Oo i) 0 Oo 0 
h. Specific students’ reading progress. 0 i) 0 0 0 
i. Using student assessment data for instructional planning 0 Oo Oo o o 
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School-wide Intervention Materials 


Use of Text Sets 


Text sets are defined hereas collections of texts at different reading levels, centered around specific content area 


themes. 
6. a) Do you use text sets to teach literacy through your content area? 
Yes (SKIP to 
No ( 
b) If you are not using text sets, please indicate why you are not using them below (Check all that apply) 
(After this item, respondents will proceedto eas) 
They have not been made available to me 
The content is not relevant/interesting to my students. 
Available text sets are not sufficiently aligned to the curriculum of my subject area. 
The range of reading levels covered by available texts sets is too high for my students. 
The range of reading levels covered by available texts sets is too low for my students. 
| did not receive text sets for the topics that we are covering in my class. 
Other (Specify): 
7. Please indicate the proportion of your students for whom the following statements are true. 
Pave Most About A few pert 
students half students y 
students students 
The range of reading levels covered by the available 
text sets is too high for these students. 
The range of reading levels covered by the available q q q q q 
text sets is too low for these students. 
The range of reading levels covered by the available 5 5 q 5 5 
text sets is appropriate for these students. 
Available text sets are relevant to their interests. oO oO oO oO oO 
A ea text sets are appropriate to their learning q q q q q 
Available text sets motivate students to learn more q 5 q q 5 
about a topic. 
Pea text sets are appropriate to their literacy q q q q q 
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8. Please describe your use of text sets to teach comprehension strategies and to provide access to 
leveled content reading materials in your typical content area classroom. Use the table below to 
indicate the units with which they are used and the duration of those units. 


Identify those units for which 
you use text sets 


1) 


2) 


3) 


4) 


5) 


Typical duration of 


___days/___ weeks 


___ days/___ weeks 


___days/___ weeks 


___ days/___ weeks 


__ days/___ weeks 


9. Rate how comfortable you are with using the available text sets to support student learning in language 


arts. 
1 2 3 4 5 
Not at all Very 
Comfortable Comfortable 
0 0 0 0 0 


10. In your opinion, how well are the availabletext sets aligned to the curriculum of your subject area? 


1 2 3 4 5 
Poorly Adequately Very well 
Aligned Aligned aligned 
go go go go go 
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Professional Development in Literacy Instructional Practices 


11. For each of the following literacy teaching practices, indicate: 


d. Did you receive professional development during the current school year? 


If So: 


e. Please indicate whether you are using this practice as part of your content area instruction, and 
f. Rate your comfort implementing each teaching practice within your content area instruction. 


a) IF YES: 
Received | h) Using as part of 
Literacy based teaching practices ie ; ; vee 
No | Yes No Yes Not at all 
Comfortable 
Building academic vocabulary o;o 0 0 o o 
Using classroom libraries o;o Oo Oo o o 
Creating literacy-rich classroom environments o;o Oo Oo 0 o 
Differentiating instruction o;o Oo Oo o o 
Explicit vocabulary instruction o;,0O 0 0 Oo 0 
Incorporating text sets in your instruction o;,0O 0 i) Oo Oo 
Increasing student motivation o;o 0 0 o o 
Supporting students’ self-directed learning o;,0O i) 0 Oo 0 
Using before, during, and after reading strategies o;o Oo Oo 0 o 
Using formal assessments to guide instruction o;o 0 Oo 0 o 
Using informal assessments to guide instruction o;o Oo 0 0 o 
Sea ea (laptops, desktops, and or gio q q q q 
Using literacy-based software o;o Oo Oo o o 
Using partner reading o;o 0 0 o o 
Using the whole-part-whole classroom instruction model o;,0O | 0 0 | im) 0 
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Respondent Information 


12. What is your position? O General Education Teacher OBilingual/ELL teacher MSpecial education 
teacher MLiteracy Enrichment Specialist Reading Specialist Other (Please 
Specify: 


13. What subject(s) do you teach? (Check all that apply) 

0 All subjects 

© Literacy/Reading/English language arts 

Mathematics 

Science 

Social studies 

Other: (Please Specify: ) 


14. At which grade level(s) are you teaching this year (2009-10)? (Check all that apply): 


0k010203040506070809 010 011 012 


15. What is the name of your school? [drop down list] 
Aldridge 


Carnegie 


Casals 


Clark 


Dubois 


Dvorak 


Emmet 


Esmond 


Gregory 


Henderson 


Madison 


Mann 


McKinley 


Mireles 


Morgan 


O'Keefe 


Otis 


Parkman 


Pasteur 


Pullman 


Reinberg 


Schiller 


Sexton 


Spry 


Swift 


Turner- 
Drew 


Wacker 


Walsh 


Whistler 
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16. How many years have you been teaching? [INSERT TEXTBOX] 
17. How many years have you been teaching at this school? [INSERT TEXTBOX] 


18. How many years have you been teaching in your subject area? [INSERT TEXTBOX] 


Teachers will be reimbursed by CPS-Striving Readers for their time to complete this survey. In order to be reimbursed we need 
you to identify yourself so that we can verify that you completed the survey. If you would like to be reimbursed, please provide 
your name and email address below, and be sure that you identified your school in the previous item. Your survey responses 
will still remain strictly confidential and will never be reported in any form that would allow anyone to connect your responses with 


your name. Providing this information is optional. 


Your Name: Email: 


Thank you for completing this survey! 
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CPS Striving Readers Evaluation — Spring 2010 Principal Interviews — Treatment Schools 


Metis Associates 

90 Broad Street 

Suite 1200 

New York, NY10004 

www.metisassociates.com 

Questions, contact: Artis Bergman, Research Analyst 
abergman@metisassociates.com or 212.425.8833 


Chicago Public Schools Striving Readers Evaluation 


Spring 2010 Principal Interviews — Treatment Schools — Copy of Questions for Interviewees 


Note: For this study, we are surveying and interviewing staff both from schools that are using the Striving Readers 
curriculum, and those from a comparison sample of schools that are not using Striving Readers. We are interested in 
learning about the literacy interventions for grades 6 through 8, so please respond to all of the questions as they 
relate to those grades only. Since we are requesting a lot of information and we have a limited period of time, most 
of the questions will be short-answer; however, please feel free to comment on any questions that you would like to. 
You will also have an opportunity to elaborate further at the end of the interview. 


This interview will take about 30 minutes. Your responses will be kept confidential. We don't identify individual 
respondents or their schools. We also may ask to tape the interview to ensure it has been recorded accurately. You 
may decline to be recorded. 


1. Has your role in the Striving Readers Initiative changed since last year? 


OYes 
ONo (SKIP TO Q2) 


a. Ifso, how? 
Literacy Leadership Teams and Supports 
2. Does your school have a librarian? 


OYes 
ONo (IF NO, skip over librarian as response option in future questions) 


3. Does your school have a Literacy Team (a “vertical” team focusing on literacy issues across 
grade levels)? 


OYes 
ONo (IF NO, SKIP TOQ4) 


a. Which of your staff are members of the Literacy Team? 


0 Principal © Grade level teacher(s) 0 Literacy Intervention Teacher 


0 Librarian(s) © ELL/ESL Teacher(s) © Special education teacher(s) 
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CPS Striving Readers Evaluation — Spring 2010 Principal Interviews — Treatment Schools 


Oj Other: 


b. How often does the Literacy Team meet? 


OHas not met OLess than once per month OOnce per month 


OBiweekly OWeekly Several times a week or more 


c. Overall, rate the quality of the literacy team’s performance in the following areas. 


Poor Fair Good Excellent | Not Sure 
Addressing the needs of all students im) Oo q q qo 
Addressing the needs of struggling q q q q q 
readers. 
Addressing the needs of grade-level + 5 q 4 4 
teams. 
Addressing the needs of individual q q q q 5 
teachers 
Addressing school wide needs (grades 6- 
8) included in SIPAAA o = S et 
Using assessment data and or student q q q q q 
work to drive instruction 
Supporting vertical and horizontal teacher q q q q q 
collaboration 
Improving literacy instruction at your q qq 7 5 q 
school. 


4. Does your school have grade level teams (“horizontal” teams consisting of staff across subject 


areas from the same grade)? 


OYes 
ONo (IF NO, Skip to Q5) 


a. Which of your staff are members of the grade level teams? 


Principal OELA teacher(s) 

Content area teachers 0 Literacy Intervention Teacher 
OLead Literacy Teacher © ELL/ESL Teacher(s) 

0 Special education teacher(s) OLibrarian(s) 

0 Other: 


b. How often do the grade level teams meet? 


OHas not met OLess than once per month Once per month 


OBiweekly OWeekly Several times a week or more 


A-71 


CPS Striving Readers Evaluation — Spring 2010 Principal Interviews — Treatment Schools 


c. Overall, rate the quality of the grade level team’s performance in the following areas. 


Poor Fair Good Excellent ot 
Sure 

Addressing the needs of all students o o q qo qo 
Addressing the needs of struggling readers. og o ms) q o 
Using assessment data to plan instruction og Oo im) q o 
Using assessment data to establish vertical 
and horizontal literacy goals by grade level a . a 
Improving literacy instruction at your q 7 5 q q 
school. 


Use of Assessment Data 


We would like to learn more about the use of assessment data and how that impacts instruction. 


5. In what ways, if any, is your school using student assessment data beyond mandated reporting 
to the district and state? For each of they uses of assessment data listed below, please 
indicatethe extent to which student assessment data in your school are used for each purpose. 


a) To what extent? 
Is student assessment data used for this Toa Toa Toa 
purpose? Not at all small moderate large 
extent extent extent 

Screening students’ ability levels for placement q q q q 
in intervention programs 
Diagnosing students’ strengths and support 
needs for placement in specific courses or Oo qo qo qo 
instructional groups 
Identifying trends in fluency and comprehension 

a o q qo qo 
abilities across groups of students 
Identifying trends in vocabulary knowledge q q q q 
across groups of students 
Monitoring overall student progress for the 
purpose of assessing success of instructional Oo q qo im) 
programs and methods 
Differentiating instruction (more than small group 
structures, this refers to providing different content 
and/or instructional techniques specifically tailored to im qo qo 0 
meet students’ individual educational needs and/or 
learning styles.) 
Planning on-site professional development Oo a) qo 0 
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Integration of Literacy Instruction in Content Areas 


6. We would like to know more about your school’s efforts to integrate literacy into the 
content areas. 


To what extent? 
To what extent do non-literacy teachers Toa Toa Toa 
integrate literacy into the content areas? pt small moderate large 
extent extent extent 
Math Oo q 0 q 
Social Studies Oo qo 0 qo 
Science Oo qo 0 q 


7. Through the Striving Readers program, all participating schools received a series of text 
sets with accompanying teacher guides. These are sets developed and distributed by the 


Striving Readers program that include collections of short books centered around 


specific content area themes, written at a variety of reading levels so students can access 


the books independently. 
Are these SR text sets being used in the content area classrooms in each subject? 
a) Social Studies 


OYes 
OWNo 
Don’t Know 


b) Science 


OYes 
OWNo 
ODon’t Know 


8. Have staff at your school other than ELA teachers been attending professional development 
for the Striving Readers project? (Includes: Bilingual, SPED, Math, Science, Social Studies 
teachers) 


OYes 71 
OWNo 


If YES: Who has received professional development? Please specify staff positions: 
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If Yes: In what topics did they receive training? (check all that apply) 


OBuilding academic vocabulary Supporting students’ self-directed learning 


OWUsing before, during, and after reading 


Using classroom libraries : 
strategies 


Using formal assessments to guide 


OCreating literacy-rich classroom environments : a‘ 
instruction 


ODifferentiating instruction (more than small group oe ; 
structures, this refers to providing different content and/or OUsing informal assessments to guide 
instructional techniques specifically tailored to meet students’ | instruction 

individual educational needs and/or learning styles.) 


ODirect vocabulary instruction Using classroom computers 
Cilncorporating text sets in your instruction OWUsing literacy-based software 
CIncreasing student motivation Using partner-reading 


Has any of your staff taken any courses towards their Masters or English language arts 
endorsement in NationalLouisUniversity as part of Striving Readers? 


0 Yes: 71 
ONo 


If Yes: Who, When? 
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Technology 


11. For each of the following technology resources, please indicate: 


a. if your teachers are using this resource to teach literacy 
b. for those resources your teachers are not using, please indicate why (Check all that apply) 


Resources Using? | If NO, why not? (Check all that apply) 
Software Students do not | Teachers do not | I do not feel that they 
Not applications have sufficient | feel comfortable | offer sufficient added Other 
Yes No | workin a : “6. 
g ave not been access to the using the benefit compared to (please specify): 
properly installed resources technology traditional media 

Media Centers 
(classroom-based fy, 
station with computers Pe a o a a e iplease speciiy 
and a printer) 
Listening Centers 
(Classroom CD or 
cassette player, read- oa oq oO mi Oo 0 (please specify): 
along audio books and 
headphones) 
Handheld 
Computers oo Oo oO Oo Oo Oo (ilesee speci: 
(Palm Pilots) 
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12. For each of the following technology resources that your teachers are using to teach literacy, 
please indicate to what extent that technology resource is integrated into the literacy 


curriculum. 
Extent integrated 
Resources NA Not at all Samawhat Thoroughly 
integrated integrated integrated 
Media Centers 
(classroom-based station with o o oO o 
desktop computers and a printer) 
Listening Centers 
(Classroom CD or cassette player, 
read-along audio books and 2 7 a S 
headphones) 
Handheld Computers q q q q 


(Palm Pilots) 


13. Overall, how well is technology integrated into the literacy curriculum? Would you say it is: 


ONot at all integrated OSomewhat integrated 


OThoroughly integrated 


Comments: 


Professional Development 


14. Have you participated in Striving Readers professional development related to literacy this 


school year or last summer? 


Oj Yes 
O No (SKIP TO Q16) 
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15. For each of the following Striving Readers professional development sessions conducted during 


the 2009-2010 school year, please indicate: 
"Whether you participated, and 
" Ifso, how useful the session(s) was (were) in helping you support student learning in 


language arts 


oe y me If YES, how useful was the session? 
; : participate? 
Professional Development Sessions 
Somewhat | Moderately | Extremely 
ms er Noe Usetal Useful Useful Useful 
Monthly Principals Meetings q q q q q q 
(Leaders Seminars) 
2009 Summer institute Oo mi) og Oo Oo q 
School-year follow-up institutes Oo im) im) Oo 0 qo 
Saturday seminars im) im) o Oo Oo q 
On site training during literacy team q q q q q q 
meetings 
School-based Striving Readers q q q q 4 q 
professional development 


Perceptions of the Literacy Curriculum 


16. Overall, what are the strengths of your school’s literacy curriculum? 


17. Overall, what are the challenges to your school’s literacy curriculum? 


18. What does your school need to better support literacy instruction? 


19. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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Metis Associates 

90 Broad Street 

Suite 1200 

New York, NY10004 

www.metisassociates.com 

Questions, contact: Artis Bergman, Research Analyst 
abergman@metisassociates.com or 212.425.8833 


Chicago Public Schools Striving Readers Evaluation 


Spring 2010 Principal Interviews — Control Schools — Copy of Questions for Interviewees 


Note: For this study, we are surveying and interviewing staff both from schools that are using the Striving Readers curriculum, and those from a comparison 
sample of schools that are not using Striving Readers. We are interested in learning about the literacy interventions for grades 6 through 8, so please respond to 
all of the questions as they relate to those grades only. Since we are requesting a lot of information and we have a limited period of time, most of the questions 
will be short-answer; however, please feel free to comment on any questions that you would like to. You will also have an opportunity to elaborate further at the 
end of the interview. 


This interview will take about 30 minutes. Your responses will be kept confidential. We don't identify individual respondents or their schools. We also may ask 
to tape the interview to ensure it has been recorded accurately. You may decline to be recorded. 


1. Has your role in supporting the literacy instruction that takes place at your school changed since last year? 


OYes 
ONo (SKIP TO Q2) 


a. Ifso, how? 
Literacy Leadership Teams and Supports 
2. Does your school have a librarian? 


OYes 
ONo (IF NO, skip over librarian as response option in future questions) 


3. Does your school have a Literacy Team (a “vertical” team focusing on literacy issues across grade levels)? 
y Literacy leam g y g 
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OYes 
ONo (IF NO, SKIP TO Q4) 


a. Which of your staff are members of the Literacy Team? 


O Are youamember? 0 Grade level teacher(s) 0 Reading Specialist 
0 Librarian(s) O ELL/ESL Teacher(s) 
© Special education teacher(s) Oj Other: 


b. How often does the Literacy Team meet? 


OHas not met OLess than once per month OOnce per month 


OBiweekly OWeekly Several times a week or more 


c. Overall, rate the quality of the literacy team’s performance in the following areas. 


Poor Fair Good Excellent | Not Sure 
Addressing the needs of all students im) q o 0 im) 
Addressing the needs of struggling readers Oo oq o 0 o 
Addressing the needs of grade-level teams Oo qo Oo o o 
Addressing the needs of individual teachers im) mi) o o im) 
Addressing school wide needs (grades 6-8) 
included in SIPAAA a . 7 a a 
Using assessment data and or student work q q q q q 
to drive instruction 
Supporting vertical and horizontal teacher q q q q q 
collaboration 
Improving literacy instruction at your q q q q q 
school 
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4. Does your school have grade level teams (“horizontal” teams consisting of staff across subject areas from the same grade)? 


Yes 
ONo (IF NO, Skip to Q5) 


a. Which of your staff are members of the grade level teams? 


OAre you a member? 


OELA teacher(s) Content area teachers Reading Specialist 
OELL/ESL Teacher(s) OSpecial education teacher(s) 
OLibrarian(s) Other: 


b. How often do the grade level teams meet? 


OHas not met OLess than once per month OOnce per month 


OBiweekly 0 Weekly Several times a week or more 


c. Overall, rate the quality of the grade level team’s performance in the following areas. 


Poor Fair Good Excellent | Not Sure 
Addressing the needs of all students o o q q qo 
Addressing the needs of struggling readers qo Oo im) q qo 
Using assessment data to plan instruction o Oo mi) q q 
Using assessment data to establish vertical 4 4 q q q 
and horizontal literacy goals by grade level 
Improving literacy instruction at your school o Oo mi) q q 


Use of Assessment Data 


We would like to learn more about the use of assessment data and how that impacts instruction. 
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5. In what ways, if any, is your school using student assessment data beyond mandated reporting to the district and state? For each of 
the uses of assessment data listed below, please indicate the extent to which student assessment data in your school are used for each 


purpose. 


a) To what extent? 


Is student assessment data used for this Toa Toa Toa 
purpose? Not at all small moderate large 
extent extent extent 

Screening students’ ability levels for placement q q q q 
in intervention programs 
Diagnosing students’ strengths and support 
needs for placement in specific courses or q o q im) 
instructional groups 
Identifying trends in fluency and comprehension 

ae q o q o 
abilities across groups of students 
Identifying trends in vocabulary knowledge q 7 q q 
across groups of students 
Monitoring overall student progress for the 
purpose of assessing success of instructional q Oo qo im) 
programs and methods 
Differentiating instruction (more than small group 
structures, this refers to providing different content 
and/or instructional techniques specifically tailored to im) mi qo 0 
meet students’ individual educational needs and/or 
learning styles.) 
Planning on-site professional development im) 0 q o 
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Extended Day Literacy Program 


6. a) Does your school have an onsite, after-school or before-school intervention program? 


OYes, 
ONo (Skip to Q8) 


b) Who is the program targeting? 

Struggling Readers Only 

0 Students at or Above Grade Level (Skip to Q8) 
0 All Students, including struggling readers 


c) What criteria are used to determine students’ eligibility for the after-school or before-school literacy program? 
[Probe: test scores (specific cut-offs?); other literacy assessments (specify type and cut-off criteria); teacher referrals; etc. ] 


Criteria: 


7. a) Overall, how appropriate would you say that the after-school or before-school program is to the reading levels and needs of the 
students who are currently participating? 
0 Not at all appropriate 
O Somewhat appropriate 
O Appropriate 
O Very appropriate 


b) Overall, how effective has the after-school or before-school literacy program been in improving the literacy abilities of struggling readers? 


Not at all | Minimally | Somewhat Effective Very Don’t 
effective effective effective effective know 
| | Oo | Oo | 
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Integration of Literacy Instruction in Content Areas 


8. We would like to know more about your school’s efforts to integrate literacy into the content areas. 


To what extent do non-literacy teachers integrate literacy into the To what extent? 

content areas? Not at all To a small extent To a moderate extent To a large extent 
Math Oo 0 Oo o 
Social Studies gq 0 o Oo 
Science gq 0 o Oo 


9. Many schools have developed and use school-wide text sets for content area instruction (i.e., collections of reading materials centered 
around specific content area themes). Does your school use such text sets? 


Yes 
ONo [skip to Q11] 
Don’t Know [skip to Q11] 


a) Are these text sets being used in social studies classrooms? 


Yes a 
OWNo 
ODon’t Know 


If YES: Do the social studies text sets include books written at a variety of reading levels so students can access them 
independently? 


OYes 
OWNo 
ODon’t Know 


CPS Striving Readers Evaluation — Spring 2010 Principal Interviews — Control Schools 
b) Are these text sets being used in Science classrooms? 


Yes q 
OWNo 
ODon’t Know 


If YES: Do the science text sets include books written at a variety of reading levels so students can access them independently? 


OYes 
OWNo 
ODon’t Know 


A-84 


Technology 


10. For each of the following technology resources, please indicate: 
if your teachers are using this resource to teach literacy 


a. 
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b. for those resources your teachers are not using, please indicate why (Check all that apply) 


Using? | If NO, why not? (Check all that apply) 
Software Students do not | Teachers do not | I do not feel that they 
Yes No We do not Not applications have sufficient feel comfortable | offer sufficient added Other 
have working have not been access to the using the benefit compared to (please specify): 
Resources properly installed resources technology traditional media 
Media Centers 
(classroom-based oa q q gq q 4 q Other 
station with computers (please specify): 
and a printer) 
Listening Centers 
(Classroom CD or Other 
cassette player, read- o 09 0 0 o Oo o o ; 
along audio books and (please specify): 
headphones) 
Laptop computers o 09 Oo Oo 0 o im) im) Other ; 
(please specify): 
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11. For each of the following technology resources that your teachers are using to teach literacy, 
please indicate to what extent that technology resource is integrated into the literacy 


curriculum. 
Extent integrated 
Resources NA Not at all pean aor Thoroughly 
integrated integrated integrated 
Media Centers 
(classroom-based station with o o Oo o 
desktop computers and a printer) 
Listening Centers 
(Classroom CD or cassette player, 
read-along audio books and 7 2 7 
headphones) 
Laptop computers o Oo Oo im) 


12. Overall, how well is technology integrated into the literacy curriculum? Would you say it is: 


(INot at all integrated 


OSomewhat integrated 


OThoroughly integrated 


Comments: 


Other Literacy Initiatives 


13. Is your school providing any formal literacy initiatives or programs for grade 6 through 8 
students during this academic year (SY 2009-2010)? 
[Probes: Names of programs (e.g. Read180) if using formal programs; Groups of students targeted by 
the initiatives (especially lower readers); Whether the initiative is implemented during the school day 
or after school; Primary instructional strategies and techniques involved] 


Perceptions of the Literacy Curriculum 


14. Overall, what are the strengths of your school’s literacy curriculum? 


15. Overall, what are the challenges to your school’s literacy curriculum? 


16. What does your school need to better support literacy instruction? 


17. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and am one of the members of the Striving Readers evaluation team 


at Metis. We are interviewing members of the SR district leadership team to learn more about the 


program; this interview will take about 75 minutes. I would like to tape this interview to be sure I have 


recorded it accurately. Is this all right? 


1. What is your role as the Project Director of the Striving Readers program? 
a. Have your responsibilities changed during the past three years? If so, how? 
b. What is your role in monitoring program implementation? 


c. What other responsibilities do you have (separate from Striving Readers)? Has the extent or 
type of these responsibilities changed over time? How do these responsibilities impact your 


ability to fulfill your role in SR? 


2. Please describe your work with the following key Striving Readers staff. Has your work with 


these staff changed over time? If so, how? 

a. District coordinators. 
= How do you communicate with them? 
«How often do you meet with each coordinator individually? As a group? 
«How do you determine individual responsibilities on the project? 
= Describe how you ensure accountability for the coordinators accountable? 
=" What types of support do you provide to district coordinators? 

b. Senior Literacy Consultant 
= How do you communicate with her? 
=" How often do you meet with her? 
= What is the nature of the collaboration? 

c. Director of Reading and Language Arts (Paul Whitsitt) 
=" How do you communicate with him? 
=" How often do you meet with him? 
= What is the nature of the collaboration? 

d. Individual schools. 
=" How often do you visit the schools? 


=" With whom do you meet (e.g., principals, LITs, teachers, students, librarian, tech 


coordinator, other)? 
= What is the nature of the collaboration? 
=" Do you conduct observations of instruction? 


= Have you had sufficient opportunity to follow up on these visits to the extent you 


would like? 


3. Haveany district level changes and/or restructuring affected program implementation? 
How are district SR leadership adapting to these changes? 


4. Describe the support you receive from the district for this program. 
a. Has the support changed over time? If so, how? 
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5. How have school-based changes and/or restructuring affected program implementation? (Probe 
for: changes in school administrations, changes from self-contained to departmentalized structure.) 
a. How are district SR leadership adapting to these changes? 


6. To what extent has the district SR leadership been able to convey expectations of accountability 
for program implementation? How has this been accomplished? 


e Have recent district-level changes affected this process? 
e What impediments, if any, have been encountered to establishing accountability? 


e To what extent do you believe that school administrators are communicating these expectations 
and holding their staff accountable? 


7. We would like to learn more about the District-Level Team’s use of data to inform project 
management. In what ways, if any, does the district-level team use the following types of data to 
inform project management? (Probe for: rubrics, surveys, implementation/evaluation data, 
professional development, assessment, and other, data.) 

a. Has the team used these data to make adjustments to address implementation challenges? 
Please provide examples. 


8. Now I would like to ask some questions about the professional development plan for Year 4 (SY 
2009-2010). 

a. Did you personally facilitate professional development activities during year 4? If so, how 
did this fit into the overall PD plan for SR? 

b. Considering the overall PD plan for Year 4, in what ways, if any, was professional 
development differentiatedfor different schools and different staff? 

c. Have there been any changes over time? If so, why? 

d. How many new teachers were in Striving Readers schools in Year 4? Were they offered 
specific professional development opportunities (or will they be in Yr 5)? 

e. What were the successes of the professional development activities of Year 4? 

f. What challenges did you encounter with the implementation of professional development 
activities in Year 4? How were these challenges addressed? 


9. In what ways has the Chicago Striving Readers program used each of the following types of 
technology as a tool to help improve differentiated literacy instruction for readers at all levels? 
What have been the successes and challenges of these efforts? To what extent have schools succeeded 
in using these technologies to support differentiated instruction? 

e Classroom media centers! 
e Listening centers” 
e Handheld computers (Palm Pilots) 


10. To what extent has program implementation expanded into upper grades and into other subject 
areas in the past year? Please describe factors facilitating or hindering this process.(Probes: role of 
SR text sets (developed and distributed by SR), PD for grade 7 and 8 teachers and for non-literacy 
staff, variations among schools) 


'3 computers and a printer 
“students access models of fluency, record and listen to themselves reading,and listen to audio books 
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11. Describe the major successes of the Striving Readers program in Year 4 (SY 2009-2010). When 
possible, please differentiate between the blended model, the targeted intervention and the 
intensive intervention. 


12. What are some of the challenges that have been encountered in Year 4 (SY 2009-2010)? When 
possible, please differentiate between the blended model, the targeted intervention, and the 
intensive intervention. 

a. How have these challenges been addressed or how will they be addressed in Year 5? 


13. Is there anything else you would like to add regarding the program or literacy activities for 
struggling readers in the district? 
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Interviewee Name: Donna Ogle Interviewee Title: 

Date: Start Time: End Time: 
Interviewer: 

Introduction: I'm ; 1am one of the members of the Striving Readers evaluation team at 


Metis. We are interviewing members of the SR district leadership team to learn more about the program; 
this interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. Is this all right? 


1. What is you role as Senior Literacy Advisor of the Striving Readers program? 
a. Have your responsibilities changedduring the past four years? If so, how? 
b. Describe your role in project-level components of Striving Readers (Probe for: 
participation in summer institutes; 
participation in Saturday seminars; 
coordination of coursework with National-LouisUniversity; 
establishing goals and identifying and addressing challenges to meeting them). 
Have these roles changed during the past four years? If so, how? 


2. Please describe your work with the following key Striving Readers staff. 
a. Project director. 
= How do you communicate with her? 
= How often do you meet with her? 
= What is the nature of the collaboration? 
= Ifat all, how does your work with the Project Director strengthen project leadership? 
b. District coordinators. 
= How do you communicate with them? 
=" How often do you meet with them? 
= What is the nature of the collaboration? 
c. Individual schools. 
=" How often, if at all, do you visit the schools? 
=" How often, if at all, do you work directly with the principals of the schools? 
= With whom else do you meet (e.g., teachers, students, librarian, tech coordinator, 
other)? 
=" Do you conduct observations of instruction? 
= What is the nature of the collaboration? 


3. We would like to learn more about the District-Level Team’s use of data to inform project 
management. In what ways, if any, does the district-level team use the following types of data to 
inform project management? (Probe for: rubrics, surveys, implementation/evaluation data, 
professional development, assessment, and other, data.) 

a. Has the team used these data to make adjustments to address implementation challenges? 
Please provide examples. 


4. What progress do you feel the project has made in moving beyond small group instruction into 
authentic use of differentiated instruction? What successes and challenges have been encountered 
in these efforts during Year 4? 
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5. Describe the major successes of the Striving Readers program in Year 4 (SY 2009-2010). When 


possible, please differentiate between the blended model, the targeted intervention and the 
intensive intervention. 
a. Do you feel that the program is reaching maturity? How do you think “maturity” should be 
defined? 


What are some of the challenges that have been encountered in Year 4 (SY 2009-2010)? 
a. How were these challenges addressed last year and how will they be addressed in year 5? 


What do the schools need to better support literacy instruction? 


Is there anything else you would like to add regarding the program or literacy activities for 
struggling readers in the district? 
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Interviewee Name: Date: 
Interviewer: Start Time: End Time: 
Introduction: I'm and am one of the interviewers with the Chicago Public Schools 


Striving Readers evaluation team at Metis. We are interviewing members of the SR district leadership 
team to learn more about the program; this interview will take about one hour. I would like to tape this 
interview to be sure I have recorded it accurately. Is this all right? 


1. What is you role as a District Coordinator of the Striving Readers Program? 
a. Have your responsibilities changed over time? If so, how? 
b. What training have you received to help you execute these responsibilities? 


2. Please describe how you work with the individual schools. 
a. How often do you visit each school? 
b. With whom do you primarily work? What is the nature of your collaboration? Has your work 
with the following stakeholders changed over time? If so, how? 
7 School administration 
«Teachers 
« Literacy Intervention Teacher 
« — Students 
" Librarian 
« School technology coordinator 
7 Other? 


3. To what extent (and in what ways) do you communicate SR expectations to school 
administrators and teachers and ensure accountability for the implementation of Striving 


Readers? (Probes: focus is as much on how they define the expectations as on how they communicate 


them. How do they monitor program implementation at the school level? The classroom level?) 


4. Describe the support you receive from the district for this program. 
a. Has the support changed over time? If so, how? 
b. Do you have other non-SR competing priorities? If so, how do they affect your SR work? 


5. What supports do you provide to the school? (Probes: support of data collection and analysis 
processes; collaboration with LITs; role in school literacy team meetings; what types of on-site PD 
do you offer and to whom?) 


6. Have any district level changes and/or restructuring affected program implementation? How? 
7. How have school-based changes and/or restructuring affected program implementation at the 


school level? On the program as a whole? (Probe: changes in school administrations, changes 
from self-contained to departmentalized structure) 
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8. We would like to learn more about the leadership team responsible for implementation of 
Striving Readers at the district level. 
a. Who comprises this district-level team? 
b. How often do you meet with other District Coordinators? With the Project Director? With the 
Literacy consultant [Donna Ogle]? 
c. What is the nature of this collaboration? 
= What topics do you discuss? 
" How does the district-level team respond to challenges to implementation of the 
Striving Readers program? Please provide some examples. 


9. What types of data, if any, do you and/or the district team use to inform your work? (Probe for: 
implementation, professional development, student assessment, evaluation data) 
a. Please explain how and for what purposes you use each type of data. 


Next, we would like to know more about this year’s implementation of SR in grades 7 and 8, in non- 
ELA classrooms, and in schools of different sizes. 


10. To what extent is SR being implemented in the seventh and eighth grades? 
a. Does this vary by school and/or teacher? Please describe factors facilitating and/or hindering 
this process. 
b. Has the expansion of the SCRMA initiative affected the extent of implementation? 


11. To what extent are non-literacy teachers integrating literacy instruction into their content areas 
as part of the Striving Readers program? 
a. Does this vary by school and/or teacher? Please describe factors facilitating and/or hindering 
this process. 
b. Has the expansion of the SCRMA initiative affected the extent of integration? 


12. Have you observed any differences in program implementation between smaller and larger 
schools in each of the following areas: 

Work of the LIT (e.g., grade-levels with which they work, intensity, collaboration with 

teachers) 

Availability of staffing such as literacy coaches, librarian or technology coordinator 

Structures such as grade-level teams and/or literacy team 

Intersection between SCRMA initiative and Striving Readers 

Other 


ones 


13. What have been the major successes and challenges in utilizing technology to improve 
differentiated literacy instruction? (Probe: availability of hardware, software, teacher comfort 
level; use to support differentiated instruction, not just small group instruction.) 

a. Handheld Computers (Palm Pilots) 
b. Media Centers 
c. Listening Centers 


14. Describe the major successes of the Striving Readers program in Year 4. When possible, please 
differentiate between the blended model, the targeted intervention and the intensive 
intervention. 

a. What district-level, school-level and classroom-level factors facilitate the implementation and 
success of the program at each of these levels? 
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15. What are some of the challenges that have been encountered in Year 4? When possible, please 
differentiate between the blended model, the targeted intervention and the intensive 
intervention. 

a. How have these challenges been addressed or how will they be addressed in year 5? 


16. What do the schools need to better support literacy instruction? 


17. Is there anything else you would like to add regarding the program or literacy activities for 
struggling readers in the district? 
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Interviewee Name: Title: 
Interviewer: Date: Start Time: End Time: 
Introduction: I'm ;] am one of the members of the CPS Striving Readers evaluation 


team at Metis. We are interviewing members of the SR district leadership team to learn more about the 
program; this interview will take about one hour. I would like to tape this interview to be sure I have 
recorded it accurately. Is this all right? 


1. Whatis your role as [Technology Coordinator/Technology Consultant] of the Striving Readers 
program? 
a. Have your responsibilities changed over time? If so, how? 
b. How does your role relate to that of the other Technology Co-Coordinator [Rob 
Residori/Lamarr Wilson]? Please describe how you work together. Have these roles 
changed since last year? 


2. Please describe the technology support provided to individual schools. 
a. How often do you visit each school? 
b. Who do you primarily work with? What is the nature of your collaboration? 
« Principals 
«Teachers 
» — Students 
" Librarian 
=" School technology coordinator 
7 Other? 
c. Apart from your own services, do the schools receive any other technology-focused support? 
d. Are different levels of support provided for schools that previously had no technology? 


3. Do you differentiate your school support and or professional development services? 
a. Ifyes, at what level (e.g., individual, staff type, grade, cluster, focus area)? 
b. How do you determine school or teacher needs? 
c. Has the shift from a self-contained to a departmentalized structure in the six through eighth 
grades impacted how you provide technology support? 
d. How many new teachers were in Striving Readers schools in Year 4? Were they offered 
specific professional development opportunities (or will they be in Yr 5)? 


4. Describe the support you receive from the district for this program. 
a. Has the support changed since over time? If so, how? 


5. In what ways is the Chicago Striving Readers program using Handheld computers (Palm 
Pilots) as a tool to help improve differentiated literacy instruction for the following groups of 
students? Please respond to each of the questions below and indicate how the status differs for 
each of the following groups: 

e Whole class/blended instruction model (all students) 
e In-class targeted intervention for Tier 2 and 3 students 
e After-school Intensive intervention for Tier 3 students 


For each of the above groups (where applicable)... 
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CPS: Striving Readers 
TECHNOLOGY COORDINATOR INTERVIEW PROTOCOL, Spring 2010 


[Probe for each: What have been the successes and challenges of these efforts? How have these 
successes and challenges changed over time? Provide examples.] 


a. To what extent are Palms used specifically to implement SR frameworks, strategies and 
instructional methods? 
[Probe: How are students using them (e.g. completing projects)? How are teachers using 

them (e.g. designing lessons, reviewing student work, assessment) ?] 

b. To what extent are Palms used to monitor student performance or teacher effectiveness in 
literacy instruction? Please describe these processes. 

c. What evidence is there that use of the Palms increases student motivation and engagement in 
these activities? 
What evidence is there that use of the Palms improves student learning? 

e. What evidence is there of teachers’ and/or LITs’ preparedness and comfort level using the 
Palms for these activities? What specific factors facilitate or hinder staffs use of the Palms? 

f. What evidence is there of school level administrative support for use of the Palms? What 
form does this support take, where it exists? 


6. What is the status of the following features of the Palm Pilots? 
a. Teachers and students uploading their usage data to the server. How is this feature being 
used to further augment literacy instruction? 
b. Automatically syncing Palms with desktop computer 
c. Using the palms for wireless internet access for teachers and students 
d. Designing more usable literacy based lesson plans that integrate handhelds or desktops in 
small group instruction 


7. In what ways is the Chicago Striving Readers program currently using each of the following 
additional types of technology as a tool to help improve differentiated literacy instruction for 
readers at all levels? [Probe: If/how other technologies are used to monitor student performance or 
teacher effectiveness in literacy instruction.]What have been the successes and challenges of these 
efforts? How have these changed over time? 

e Classroom media centers? 
e Listening centers* 


8. What does theStriving Readers program need to better support the use of technology to 
improve literacy instruction? 


9. Is there anything else you would like to add regarding the program or literacy activities for 
struggling readers in the district? 


°3 computers and a printer 
“students access models of fluency, record and listen to themselves reading,and listen to audio books 
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Chicago Public Schools Striving Readers Program 
Spring 2010 Case Study Observation Protocol 


INSTRUCTIONS 
The Observation data will be submitted to Metis Associates in this Word database, which has 
been pre-formatted for each section of the Observation protocol. In order to facilitate the data 
analysis process, please ensure that the following steps are followed when you enter and submit 
your data. 
>> Please re-name the Word file with the name of the school and the date of the observation, 
using the following naming convention: 


School_Grade_Subject_Day [1 or 2] Date_Consultant Initials 


>> Please be sure that a selection is made for each closed-ended item by mark an “X” in the 
appropriate box. If you select ‘Don’t Know’ for any items, please explain your response in the 
respective box. 


>> For each of the Striving Readers frameworks, strategies, and techniques that are 
indicated during the pre-observation interview, indicate in Part IV, “Specific Striving Readers 
Frameworks, Strategies and Techniques” whether that technique was observed; and if so, make 
sure to complete the corresponding description of that technique in the pages that follow. 


>> If you believe that a strategy, framework or technique was implemented that was not 
indicated during the pre-observation interview, describe the activity under “General Notes” 


>> In descriptions of observed strategies, frameworks and techniques (in Part IV), indicate 
the extent to which the indicated characteristics were present, and provide evidence/examples of 
the extent to which they were/were not present. 


>> Where possible, include comments on factors that facilitated or hindered the 
implementation of these characteristics. 


>>Specify if evidence of a characteristic was absent, not observed, or unknown 


It is very important that you do not modify this structure. Please do not add or 
remove rows, columns or pages. 


If you have any questions or problems using this database, please contact Rebecca Swann- 
Jackson at 212-430-9113 or rswann@metisassoc.com. 


A-97 


Primary Goals of the Lesson 


Specific Skills/Knowledge to be Developed 


Formal/Informal Assessment 


Other Adults (Describe.) 


SR Frameworks 


Whole Part Whole 


Independent Reading 


Small Group Instruction 


Intervention 
Reading Comprehension Strategies 


Summarizing 


Questioning 


Predicting 


Visualization 


Text structure 


Inferring 


Metacognition 


Reading Comprehension Techniques 


Marzano's Vocabulary 


PRC2 


Word Study/Word Sorting (Words Their Way) 


Interactive Read Aloud 


Reading Response 
INSERT Notes 


PLAN 
ReQuest 
KWL 


List-group-label 


Other 1 (Describe at right -->) 


Response (Enter an ‘X’ in the appropriate box) 
Yes No 


Response (Enter an ‘X’ in the appropriate box) 
Yes No 


Response (Enter an ‘X’ in the appropriate box) 
Yes No 
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Other 2 (Describe at right -->) 


Other 3 (Describe at right -->) 
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Response (Enter an ‘X’ in the appropriate 
AMPAfter-School Program box 


Achieving Maximum Potential (AMP) Program 


AMP Software 


Any Additional Notes for Question 3 Response 


If there are any additional comments on the items in the lists above 
(Question 3), please enter them at right. 


Materials Response (Enter an ‘X’ in the appropriate 
box 


: 


es 


Textbook or novel 


Text sets small books (developed and distributed by Striving Readers) 


Classroom libraries 


Vocabulary or reading response notebooks 


AMP materials 


Academic vocabulary notebooks (Marzano) 
PRC2 folders 
Graphic Organizers 


Word Sorting/Words their Way materials and notebooks 
Writing folder or notebook 


Trade novels 


Basals 


Materials for Handheld Computers (Palm Pilots) 


Other 1 (Describe at right -->) 


Other 2 (Describe at right -->) 


Any Additional Notes for Question 4 


If there are any additional comments on the items in the lists above 
(Question 4), please enter them here. 
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Instructional Technology Response (Enter an ‘X’ in the appropriate box 


Yes No 

Media centers 
Listening centers 
Word processing 
Spreadsheets 
Internet 
Digital Media (e.g., camera, video, etc.) 
Handheld Computers (Palm Pilots) 
AMP Software 
Other 1 (Describe at right -->) 
Other 2 (Describe at right -->) 

Any Additional Notes for Question 5 Response 


If there are any additional comments on the items in the 
lists above (Question 5), please enter them here. 


Brief Summary 
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Context of today's lesson 


Approximate # in Tier 1 


Approximate # in Tier 2 


Approximate # in Tier 3 


Approximate # of Special Education students 


Approximate # of English Language Learners 


Anything else we should know? 
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Results/Surprises 


Lesson Changes 


Clarifying Information 


Questions/Concerns 
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Subject 
Grade 
Class Session Start Time 


Class Session End Time 
Observation Start Time 


Observation End Time 
Number of Male Students 


Number of Female Students 


Adults in the Room Response 
Classroom Teacher 
LIT 


Others (Describe at right -->) 


***Please remember to take detailed field notes during the observation and to complete the remainder of this protocol 
only after leaving the classroom.*** 


Please provide a brief narrative account summarizing the 
lesson as observed and describe each individual activity 
in the sequence in which they occurred (include 
concurrent small group activities as separate activities). 


Response 


Narrative Account 


Be sure to address the following issues for each activity identified: 

¢ the timeframe (approximate start and end times ) and sequence in 
which they occur; 

* grouping structure(s) (whole class, small group, partners, individual 
work) 

¢ the numbers of students involved (specify whether more than one 
small group is working on the same activity) 

¢ whether the teacher, LIT and/or other adults are involved and their 
roles (lecturing, modeling, explaining, etc.) 

¢ format (presentation, discussion, silent reading, etc.) 

* types of interactions (student-student, student-teacher, LIT-teacher, 
LIT-student, etc.) 


Submit field notes, the lesson plan and any handouts separately. 
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Complete the Classroom Physical Characteristics/Management and Affective Quality section below, and the General 
Notes section that follows, based on your entire observation. Provide details as appropriate when answering 
questions if the responses differ by activity. 


Check yes or no for each question and provide clarifying Response (Enter an ‘x’ in the appropriate box) 


information as appropriate. 
Pprop Yes 


Is the daily schedule posted? 


Is student authentic work posted? 

Are there are student/teacher created charts? 
Is there a classroom library? 

Is there a listening center? 

Is there a media center? 

Are there desktop or laptop computers? 

Are there handheld computers? 


Any Additional Notes for Part Il Response 


If there are any additional comments on the items in the lists above, 
please enter them here. 


CO} N} O) Ot) RB) Ww) NM] 
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Characteristics of good implementation: Indicate the 
extent to which... 

1. Resource material related to the activities is accessible to 
groups and/or individual students? 


2. The adult is positioned so all students can easily view 
modeling and/or materials being introduced and used during 
instruction? 


3. There are sufficient books and materials for independent, 
paired, or grouped student work? 


4. There are there high interest and varied (reading level) 
reading materials for students? 


5. There is asense of order and routine? 
6. There is a respectful atmosphere? 


7. There is a purposeful interaction taking place between 
teacher and student or student and student? 


8. There are shared expectations for learning and 
achievement? 


9. Students understand the importance of content? 


10. Students collaborate and construct knowledge in 
respectful and responsive ways? 


11. Students and teachers use language that shows respect 


like “thank you”, “that was a good idea”, “I hadn’t thought of 
that”, “that is interesting”. 


12. The classroom creates a safe environment for expressing 
ideas? 


13. The teacher and students embrace cultural differences 
and honor each student’s identity? 


14. Students are interacting with peers, teachers and LITs? 


15. Students are discussing or talking about the content of 
learning activities (reading/writing)? 


16. The physical and emotional environment support 
learning? 


Evidence of extent to which characteristics are present 
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Complete the General Notes section based on your entire observation. Provide details as appropriate when 
answering questions if the responses differ by activity. 

Characteristics of good implementation: Indicate the extent to 
which... are present 

1. The teacher/LIT provides reading comprehension instruction for 

whole class with blended intervention (small group guided practice, 

differentiated work)? 


2. The teacher is engaged with instruction (not grading papers or 
otherwise occupied)? 


3. The teacher/LIT moves from whole class introduction with 
explicit instruction to small group work so students can try out the 
strategies or questioning routines in materials and with other 
students working at their own level? 


4. The teacher/LIT guides students toward reading materials that 
are individualized for different students (differentiated instruction)? 


5. The teacher/LIT activates prior knowledge and/or builds 
background knowledge with students? 


6. The teacher/LIT is assessing students during instruction formally 
or informally? 


7. The teacher and LIT work together to differentiate instruction? 


8. The teacher/LIT encourage students to make connections about 
the activity to other outside selections and/or to make personal 
connections? 


9. The teacher/LIT models the specific strategy/skill to be 
employed during the activity? 


10. The teacher/LIT introduce vocabulary and concepts in context 
and/or discussion? 


11. The teacher/LIT use writing models (or exemplars) to stimulate 
student thinking? 


12. The teacher/LIT provide explicit guided instruction at 
various times during the lesson? 


13. The teacher/LIT use different modes of activities (e.g. partnered 
activities, teacher guided instruction, independent activities) to meet 
individual student needs? 


14. The teacher/LIT or other adults meet the needs of special needs 


students who require additional support by incorporating additional 
strategies (is there a Special Education teacher available...an aide)? 
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15. The teacher/LIT assess students’ understanding of the lesson 
and task directions? 


16. The teacher/LIT provide students with opportunities to 
demonstrate learning outcomes (assessments)? 


17. The teacher/LIT provide feedback to students on progress 
made? 


18. The teacher/LIT encourage students to work with technology? 


19. The teacher/LIT use scaffolding to help students understand 
content. 


20. Students show sustained reading when reading independently? 
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Please complete the following sections by checking off all Striving Readers frameworks and techniques listed below 


that you observe during the lesson. For each area observed, go to the corresponding worksheet tab and answer the 
questions listed, briefly but specifically. 


Provide details as appropriate when answering questions if the responses differ by activity. 


Refer to the Striving Readers Implementation Handbook for additional clarification of frameworks and techniques. 


Response (Enter an ‘X’ in the appropriate box 


SR Frameworks 
Yes Don’t Know 
Whole Part Whole 


Please explain any “Don’t Know’ responses. 


Reading Comprehension Strategies 


Summarizing 
Questioning 
Predicting 
Visualization 
Text structure 
Inferring 


Metacogniton a (| 


Reading Comprehension Techniques 


Marzano's Vocabulary 
PRC2 
Word Study/Word Sorting (Words Their Way) 
Interactive Read Aloud 


Reading Response 
INSERT Notes 


PLAN oO is) 
[reas 


KWL 
List-group-label 


Please explain any “Don’t Know’ responses. 


Other 1 (Describe at right -->) 


Other 2 (Describe at right -->) 


AMPAfter-School Program Response (Enter an ‘X’ in the appropriate box) 
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| No |“ Don’t Know __ 


A-110 


Characteristics of good implementation: Indicate the 
extent to which... 

1. The teacher has organized the class such that there is 
whole group instruction at the beginning of an activity. 


2. The teacher has small group or individual activities for 
students (following) the whole group piece of the lesson. 


3. The teacher facilitates whole group instruction or follow- 
up after the small group/individual activities that is a recap of 
the initial whole group session. 


4. The teacher has clear guidelines for students regarding 
the organization of individual/small group activities. 


5. Students move easily between whole group and 
individual/small group instruction and seem comfortable with 
this organization of class activities. 


Evidence of extent to which characteristics are present 
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Characteristics of good implementation: Indicate the extent to Evidence of extent to which characteristics are 
which... present 

1. The teacher has selected a topic for mini-lesson based on student 

needs/interests and curriculum goals. 


2. The teacher monitors and analyzes students’ reading records. 


3. The teacher confers with students during independent reading 
session to teach and reinforce skills and strategies. 


4. The teacher assesses students using running records. 


5. The teacher assists students with the routines of independent 
reading. 


6. The teacher gives book talks to heighten engagement and 
motivation for reading. 


7. The teacher concludes each session with whole group sharing 
that evaluates independent reading session and ties student reading 
back to mini-lesson focus. 


8. The teacher establishes procedures and routines for independent 
reading. 


9. The teacher schedules a consistent time for independent reading. 


10. The LIT assists in identifying students’ independent reading level. 


11. The LIT administers formative assessments for Tier Three 
students during independent reading time. 


12. The LIT guide students in selecting appropriate texts for 
independent reading. 


13. Students read accessible texts. 


14. Students are actively engaged in reading and in responding to 
what they have read. 


15. The classroom library that contains a variety of genres and a 
topics and titles at various reading levels, including graphic novels, 
magazines, newspapers, and other media. 


16. Student self-selected reading materials and response notebooks 
conveniently housed. 


17. Teacher actively engages students in conversation about books 
in which reading strategies and student self-evaluation of independent 
reading process are included. 
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Characteristics of good implementation: Indicate the 
extent to which... 
1. There is a sufficient quantity of leveled narrative and 
expository texts matched to the students’ independent and 
instructional levels. 


2. The teacher/LIT introduces the text, consistent with the 
meaning, language, and visual information in the text, and the 
knowledge, experience, and skills of the reader. 


3. The teacher/LIT interacts with students individually 
observing strategy use, difficulties and successes with 
problem-solving attempts. 


4. The teacher/LIT returns to the text for one or two 
teaching opportunities to demonstrate how a reader constructs 
meaning from text, makes personal connections with text, and 
goes beyond text. 


5. Students engage in pre-reading conversation about text. 
6. Students read a text or part of a text silently or quietly. 
7. Students request problem-solving help when needed. 


8. Students revisit the text at points of problem-solving as 
guided by the teacher. 


9. Students engage in collaborative discussion about the 
text. 


10. Students engage in activities that involve extending 
understanding and responding to the text. 


11. There are a variety of instructional practices used to meet 
individual students’ needs. 


12. Students are grouped flexibly, including homogeneous 
and heterogeneous groups for different reasons at different 
times. 


13. The teacher assesses literacy performance in a variety 
of ways over time, including checklist observations of student 
literacy behaviors. 


Evidence of extent to which characteristics are present 
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Characteristics of good implementation: Indicate the 
extent to which... 

1. The teacher models what expert readers do by modeling 
before, during, and after reading literacy strategies. 


2. The teacher/Interventionist (LIT) explicitly teach literacy 
strategies and include the supporting skills of vocabulary to 
support word study in context. 


3. The teacher/Interventionist (LIT) provide whole group 
instruction, incorporating read-alouds. 


4. The teacher/Interventionsit (LIT) provide small group 
instruction by bringing students with similar needs together 
and giving them attention tailored to those needs. 


5. The teacher provides opportunities for partners/pairs to 
have time to think through their reading. 


6. The Interventionist (LIT) assesses student needs. 


7. The Interventionist (LIT) assesses, teaches/reteaches, 
practices, applies, and reassesses. 


8. Students self-monitor their reading and select strategies 
needed for comprehension. 


9. Students participate with others in their differentiated 
groups. 


10. The interventionist (LIT) is working one-on-one with a 
student. 


11. The interventionist (LIT) is weaving in and out of students’ 
activities. 


12. There are flexible grouping of students relative to student 
needs and types of class activities. 


13. The classroom seems to be a collaborative setting where 
students monitor and discuss their own progress. 


Evidence of extent to which characteristics are present 
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Characteristics of good implementation: Indicate the 
extent to which... 


The teacher/LIT is helping students develop reading 
comprehension strategies, including: 


summarizing 
questioning 
predicting 
visualization 
text structure 
inferring 


metacognition 


Evidence of extent to which characteristics are present 
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Characteristics of good implementation: Indicate the Evidence of extent to which characteristics are present 
extent to which... 


1. The teacher has established routines, rituals, and 
expectations to students around the 6-step process and are 
using the academic vocabulary notebooks. 


2. The teacher follows the 6 step process to ensure 
systematic, direct vocabulary instruction. (See six step 
process below and reflect on its usage in this observation) 


e Step 1: Provide a description, explanation, or example 
of the new term. 

e Step 2: Ask students to restate the description, 
explanation, or example in their own terms. 

e Step 3: Ask students to construct a picture, symbol, or 
graphic representing the term or phrase. 

e Step 4: Engage students periodically in activities that 
help add to their knowledge in their notebooks. 

e Step 5: Periodically ask students to discuss terms with 
one another. 

e Step 6: Involve students periodically in games that 
allow them to play with terms. 


3. Students utilize the 6 step process in order to build 
knowledge of academic content vocabulary. 


4. Students use their academic vocabulary notebooks to 
record and monitor understanding of content concepts. 


5. There is direct instruction on words that are critical to 
content concepts. 


6. Students are engaged in writing and drawing about words 
and concepts in their notebooks. 


7. Teachers and students are using a variety of methods to 
deepen and extend understanding of terms (e.g. comparing, 
classifying, creating metaphors, creating analogies). 


8. Teachers and students are involved in review activities 
and games, such as Jeopardy, vocabulary charades, 
$100,000 Pyramid, etc. 
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Characteristics of good implementation: Indicate the 
extent to which... 

1. The teacher/interventionist select word sorts at the 
students’ instructional level that will support and scaffold their 
word knowledge. 


2. The teacher/interventionist introduce new sorts to 
students by modeling and then guiding their work. 


3. The teacher/interventionist interact with students 
individually and in small groups to guide them and assist in 
problem solving. 


4. The teacher/interventionist monitor word study through 
observation, note-taking, and targeted instruction. 


5. Students engage in collaborative discourse with other 
students. 


6. Students engage in activities that will extend 
understanding of the word parts, word meanings, and general 
and specific word knowledge. 


7. Students engage in problem solving with the word sorts. 


Evidence of extent to which characteristics are present 
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Characteristics of good implementation: Indicate the Evidence of extent to which characteristics are present 
extent to which... 


1. Classrooms have inviting materials at a range of reading 
levels and the PRC2 routine clearly established. 


2. Classrooms have students productively engaged with 
partners comfortably using the PRC2 framework. 


3. All students are reading informational materials on a 
common theme or topic at their independent or instructional 
level. 


4. Students discuss answers to questions they have asked 
each other and use the texts as resources. 


5. Teachers move around the room observing some 
partners in depth - listening to students as they read and 
discuss and make notes about their performance. 


6. Students keep a record of pages they read with questions 
and responses, and new vocabulary. 


7. Opportunities exist for students in differentiated practice 
using features of informational texts (Structure and visual 
information), learning new vocabulary (morphology and 
concept clusters), developing fluency, and thinking critically by 
comparing and contrasting texts. 


8. Classroom is a learner-centered environment with 
opportunities for sharing across texts and learning 
experiences. 


9. There is an established routine in the classroom so 
students know exactly how to locate their materials, engage 
productively with their partner, and keep record of their PRC2 
sessions. 


10. The teacher models how to preview texts and attend to 
the Table of Contents, text resources (glossary, index, and on- 
line references), external features (headings, sub-headings, 
highlighted vocabulary and visual information) and the internal 
structure. 


11. The teacher models for students the process of preparing 
for, reading and discussing the content with a partner. 


13. The teacher or LIT provides positive feedback to students 
about their partner process. 


14. The teacher presents “mini-lessons” to refresh the 
process if students get off track. 
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Characteristics of good implementation: Indicate the 
extent to which... 


15. The teacher leads discussions at the conclusion of PRC2 
sessions in which students can share information they have 
learned, connections they have made, and questions they 
have. 


16. Students prepare for each page by reading silently, 
thinking about oral prosody, and identifying unfamiliar words 
so they can be pronounced accurately and then recorded in 
the vocabulary notebook so they can be learned over time. 


17. Students select or create questions to ask one’s partner. 


18. Students listen attentively as the partner reads orally and 
respond to the partner’s questions. 


19. Students read designated pages orally with fluency. 


20. Students participate in discussion courteously and with 
interest. 


Evidence of extent to which characteristics are present 
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Characteristics of good implementation: Indicate the Evidence of extent to which characteristics are present 
extent to which... 


1. The teacher shows an enthusiasm for reading and 
communicates interest in reading. 


2. The teacher shares books and articles with students and 
to model reading behavior. 


3. The teacher models persistence and stamina for reading 
long and sometimes difficult text. 


4. The teacher occasionally stops to define a word, to 
wonder aloud, to ask questions, and to respond to the reading. 


5. The teacher gives students opportunities to respond to 
the reading by discussing and writing about the text. 


6. The teacher has chart paper, overhead transparencies, 
and markers to record student questions, comments, and 
ideas, especially if the same text will be used for several days. 


7. The teacher provides a format for students to keep a 
record of the read alouds in order to remember authors, 
genres, and topics for further reading and investigation. 


8. The teacher gives clear expectations of student behavior 
during the read aloud. 


9. Students seem to enjoy listening and to discuss the read 
aloud. 


10. Students explain, question, and explore ideas in what they 
have heard. 


11. Students keep a record of the read aloud along with 
responses and reactions. 


12. The teacher is respectful of all students’ cultures and 
backgrounds. 
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Characteristics of good implementation: Indicate the Evidence of extent to which characteristics are present 
extent to which... 

1. Teacher creates a safe environment where students are 

encouraged and respected. 


2. Teacher has established routines for Reading Response 
Notebook/Learning Log usage and record-keeping. 


3. Teacher explicitly instructs, models, scaffolds, and 
coaches skill lessons for writing different types of responses 
and for talking about literary elements. 


4. Teacher uses read-alouds to generate and model 
discussion. 


5. Teacher scaffolds students toward accountable talk and 
establishes routines for conversation (e.g., staying on topic, 
discussion etiquette, adherence to the text, active listening, 
and including everyone in conversation). 


6. The students demonstrate understanding of a text 
through written response and classroom discussions. 


7. The students share thoughts and opinions during whole 
group and small group discussion. 


8. The students make a judgment and support it with 
specific references to the text and background knowledge. 


9. The students actively listen to classmates and contribute 
to the conversation collaboratively. 


10. The teacher provides opportunities to use writing to 
respond in all content areas. 


11. The teacher encourages participation and collaborative 
discussion based on the writing students have completed. 


12. There is conferencing between the teacher and individual 
or small group of students. 
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Characteristics of good implementation: Indicate the Evidence of extent to which characteristics are present 
extent to which... 


1. The teacher is leading a whole group demonstration of 
the INSERT note strategy or students independently using the 
strategy with independently read text. 


2. The teacher describes the INSERT notes strategy and 
why it is helpful. 


3. The teacher selects a piece of text to demonstrate the 
strategy. 


4. The teacher thinks aloud the reasons for choosing a 
particular symbol. 


5. The teacher elicits student responses on which symbol to 
use and why. 


6. TheLIT demonstrates INSERT note strategy and work 
intensively with individual or small groups of Tier 2 and Tier 3 
students. 


7. Students use the strategy with a common text in pairs or 
teams. 


8. With common texts, students compare and discuss their 
INSERT notes. 


9. With independently read text, students write notes 
reflecting their thoughts, questions and comments directly on 
photocopied text, on 4 column charts, or on post-it notes which 
are inserted onto text. 
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Characteristics of good implementation: Indicate the 


extent to which... 

1. Students create a graphic organizer showing the major 
sections of the text being used as their reading materials 
(textbook, magazine article, etc.). Then students use this 
organizer to make notes as they read. 


2. Students write a summary of their notes and the major 
points from the graphic organizer at the conclusion of the 
summary. 


3. The teacher/LIT explains the process and models if this is 
the first use. 


4. The teacher/LIT supports students as they create their own 
organizers. 


Evidence of extent to which characteristics are present 
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Characteristics of good implementation: Indicate the extent to 
which... 
1. Teacher and LIT modeling and providing feedback in small group 
activity 


2. Teacher, LIT and students engaged in discussion about segments of 
text stemmed from questions developed by teacher, LIT or student 


3. Students applying strategy in whole class and small group instruction 


4. The teacher and LIT choose a passage of text, then designate short 
segments within the passage 


5. The teacher and LIT provide an example through modeling and 
feedback 


6. The teacher and LIT conducts first round of ReQuest activity so that 
he/she is the one to answer questions generated by the students 


7. The teacher and LIT keeps book closed during the questioning 


8. The teacher and LIT tailor ReQuest to suit the specific needs of 
students 


9. The teacher and LIT provide task cards to struggling readers with 
specific questions from text segments 


10. The teacher and LIT scaffold questioning and answering process to 
struggling readers (Tier 2 & Tier 3 students) 


11. Students watch carefully for teacher's modeling and feedback 


12. Students read passage silently. Pay attention to the information it 
contains. 


13. Students think of questions they may be asked if they were the 
respondent. 


14. Students use own words and check passage for possible answers 
15. Students keep book open while asking question. 

16. Students keep book closed while they answering question. 

17. Students listen to answers and check in text for accuracy. 

18. Students change roles — take turns in being questioner or respondent 


19. Students apply strategy during whole class or small group 
discussion 


Evidence of extent to which characteristics are 


present 
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Characteristics of good implementation: Indicate 


Evidence of extent to which characteristics are present 


the extent to which... 


1. Teacher is leading a large group introductory 
discussion of the topic to be studied and writing down 
the student contributions. 


2. Teacher stimulates disagreement and helps focus 
questions for inquiry. 


3. Students are engaged in listening to each other 
and sharing what they know and their questions. 


4. The teacher models active comprehension by 
helping students activate their prior knowledge and 
formulate questions. 

5. Teacher is scribe and records contributions. 


6. Teacher elicits questions and disagreements. 


7. The LIT sits with and assists Tier 2 & 3 students 
during large group discussion. 


8. The LIT helps students write their own KWL notes 
or enter them on their Palm. 


9. The LIT may discuss part of the text with students 
so they can access difficult sections. 


10. Students listen to each other 

11. Students contribute ideas and questions 

12. Students use their own KWL sheets to record their 
ideas and personal questions. As they read the text 


they make notes of what they learn. 


13. Students use their Palms during this activity. 
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Characteristics of good implementation: Indicate the 


Evidence of extent to which characteristics are present 


extent to which... 


1. Small groups of students are working together to 
generate a list of words related to a content area topic and 
then grouping the words into logical categories. This 
activity will take 15 to 20 minutes with the teacher and LIT 
working with the small groups. 


2. The teacher selects a main topic or concept from a 
content area reading selection and models the process by 
brainstorming and charting with the class words that are 
related to the topic. 


3. The LIT is working intensely with the tier 2 & 3 
students to support the grouping and labeling process. 


4. Students are working in teams of three to four, 
students join together related terms from the brainstormed 
list based on common features. 


5. Students generate a descriptive title, or label for the 
collections of related terms. 


6. Following the reading of the pre-selected content area 
text, students eliminate any of the terms or groups that do 
not match the concept’s meaning in the context of the 
selection or add new terms or groups as needed. 
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1. 


2. 


Characteristics of good implementation: Indicate 
the extent to which... 
The teacher provides strategic tutoring. 


The teacher teaches one comprehension strategy 


at a time directly and explicitly. 


3. 


The teacher teaches strategies intensely and 


systematically. 


A. 


The teacher provides intensive writing 


opportunities. 


5. 


The teacher provides successful learning by 


delivering the direct instruction, practice, and 
evaluation students need to achieve maximum 


success. 
6. The teacher utilizes the technology component. 
7. Students seem to give full attention and effort. 
8. Students demonstrate a cooperative learning 


attitude toward fellow students and teacher. 


9. 


Students stay on task and follow directions. 


10. Students are engaged with reading strategies. 


11. The teacher uses small group and differentiated 
instruction. 


Evidence of extent to which characteristics are present 
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CPS Striving Readers 
Case Study — Fall 2009 School Visits 
Self-Contained Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you may know, your school was among six schools that 
were selected for an in-depth case study in part because they were identified by the district as being 
successful in their implementation of the Striving Readers initiative. We are interested in learning about 
the implementation of this program in your school, overall and for each program component, identifying 
best practices, and gaining a better understanding of the facilitating conditions and challenges to 
implementation. This interview is designed to gather information parallel to what was discussed in the 
prior two program years, but focusing on program plans and activities for the current school year. Since 
we are requesting a lot of information and I know you have a busy schedule, please be as succinct as you 
can in your answers. You will have an opportunity to elaborate further at the end of the focus group. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 
recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 
specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to yourself or your school. Is this all right? 


1. Please introduce yourselves and tell us how long have you been in this school, and which grade 
level(s) you teach this year. 


2. We would like to know about your use of specific Striving Readers instructional 
frameworks, techniques and strategies. 


a. How comfortable are you with the different frameworks, techniques and strategies identified 
in the Striving Readers grant? When did you start using them? 


Probe: 


o Striving Readers instructional frameworks (for example: whole part whole 
instruction, differentiating instruction, independent reading, small group 


instruction) 


Oo Core Comprehension Strategies (i.e., Summarizing, Visualizing, Questioning, 
Metacognition, Inferring, Predicting, Text structure) 
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o Core Comprehension Techniques (for example: Marzano’s vocabulary, PRC2, 
KWL, Word Study/word sorting, Read Aloud/Think Aloud, Reading Response, 
INSERT notes, PLAN, ReQuest, List-Group-Label, etc.) 


b. Which ones have you found to be the most successful and why? Which ones have presented 
the greatest challenges and why? Please provide examples. 


3. We would like to know more about the process that you, teachers and students use to select 
appropriate student reading materials. 


a. Do you use fluency snapshots and interest inventories to guide students’ selection of reading 
materials for their wide reading? Please provide examples. 


b. Are student reading levels used to help select appropriate materials? Please explain and 
provide examples. 


c. To what extent do students know how to select materials for wide reading that are appropriate 
to their abilities? Please explain and provide examples. 


Next, I would like to ask you a few questions about the specifics of when you meet with other 


teachers to plan and coordinate instruction and student groupings for the purpose of providing 
differentiated instruction that is appropriate to each student’s needs. 


4. Does your school have Grade-Level Teams? LlYes LINo (IF NO, SKIP TO QS) 


a. Are any of you involved in these teams? (IF NO, SKIP TO Q5) 


b. What is your role in these teams? What topics are discussed in these meetings? 


5. Does your school have a Literacy Team? LlyYes LINo (IF NO, SKIP TO Q6) 
a. Are any of you involved in this team? LlyYes LINo (IF NO, SKIP TO Q6) 


b. What is your role in this team? What topics are discussed in these meetings? 


6. We would like to know more about your collaboration with the Literacy Intervention Teacher 
and how you plan together to provide blended, differentiated instruction. 


a. In which setting(s) do you meet or collaborate with the LIT? 
O One-on-one meetings: 
Oo Grade-level team meetings: 
oO Literacy team meetings: 


b. Do you collaborate with the LIT in scaffolding instruction (modeling strategies of 
comprehension, vocabulary and fluency while supporting steadily increasing levels of 


independence)? Please provide examples. 


c. Do you plan together with the LIT for blended instruction that differentiates small group 
work? Please provide examples. 
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0 Do you collaborate with the LIT to use assessment or other data to fine-tune 
differentiated instruction? Please provide examples. 


d. Do you plan together with the LIT to ensure that Tier 2 and 3 students receive appropriately 
targeted, differentiated instruction during in-class targeted intervention sessions and AMP 
after-school lessons? Please provide examples. 

e. Have there been any changes in the strategies, activities and resources that you use during 
these push-in sessions, as compared to last year? 

oO If yes, please describe. 
Oo Why did these changes come about? 


f. What challenges, if any, have you encountered while working with the LIT and/or Tier 2 and 
3 students this year? 
Oo How were or will these challenges be addressed? 


7. We would like to learn more about your collaboration with the District Coordinator. 
a. Do you meet with him/her? If so, how often do you meet with him/her? 
__ Oncea ____ Several ____ Once ___ Several ____ Daily or 


month or less times a month a week times a week almost daily 


b. In what ways do you collaborate? What topics do you discuss? In what ways does he/she 
support your implementation of Striving Readers? 


c. Is there any additional support they could provide you with? 
8. In what other ways, on your own or in collaboration with the LIT, do you use student data? 
(This might include assessment data as well as other types of student data such as demographic 
and behavioral records.) 


a. Which kinds of data are you using? 


b. In which setting(s) (e.g., individually, as part of grade-level teams, as part of the Literacy 
Leadership Team, other) do you use these data? 


c. For what purposes? Please describe and provide examples. 


Probe & request examples: 

oO Plan for small group activities, 

0 Monitor students' success in learning techniques for developing comprehension and using 
reading strategies appropriately, 

0 Select appropriate materials at students' independent and instructional levels. 


We would like to know more about your efforts, if any, to integrate literacy into other content area 
instruction. 


9. When, if at all, did you start integrating literacy into your instruction in other content areas? 
(Probe for: this year, last year, before Striving Readers began?) 


10. Have you been able to use any of the specific frameworks, techniques and strategies identified 
in the Striving Readers grant in other content areas? 
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a. Ifnot, how have you approached integrating literacy in other subjects? 


b. Ifyou have used specific Striving Readers frameworks, techniques and strategies, please 
provide examples. (See probes for Q2a.) 
11. We would also like to know more about the types of support that you have received to help you 
integrate literacy instruction into other content areas. 


a. Have you met with any literacy experts to specifically discuss the integration of literacy into 
other content area instruction? How often? What did you discuss with them? [Probe for: 
Literacy Intervention Teacher, Lead Literacy Teacher/Literacy Coach, Striving Readers 
Coordinator, other] 


b. Have you ever participated in any professional development conducted by the Striving 
Readers program related to integrating literacy into other content area instruction? 


LlYes LINo 


oO If yes, when/what year did you start participating in this type of trainings? 

oO Which trainings did you participate in? (Probe and request specific examples: site-based 
professional development, Summer Institute and follow-up Institutes, Saturday Seminars) 

oO Which trainings were most useful and why? Which trainings were least useful and why? 

O What areas or topics would you like to receive additional support or training in? 


c. Does your school have Striving Readers text sets (i.e., sets of reading materials of different 
structures and levels, centered around specific content area themes, designed to improve 


student literacy in other subject area classes)? L] Yes LI No 


If yes, are the text sets being used in the content area classrooms? 


0 Social Studies L] Not Used—why? [LJ] Used—how? L] Don’t Know 
O Mathematics L] Not Used—why? LL] Used—how? L] Don’t Know 
O Science L] Not Used—why? [LJ] Used—how? LJ] Don’t Know 


d. What other kinds of support are available to help you integrate literacy into your content area 
instruction? 


e. What other kinds of support would you need to help you integrate literacy into your content 
area instruction? 


We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 


12. One of the components of Striving Readers is building your classroom libraries. 
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a. How do you use the classroom libraries? 


Probe & request examples: 


For independent reading? 


For small group instruction? 


To support content area instruction? 


For read alouds? 


b. Do you use interest inventories to help students self select reading material and to guide your 
purchases? Please describe the process. 


c. How do you organize books in your classroom library? (Probe: Is there more than one 
criterion used to organize the libraries?) 


d. Have you or your students encountered any challenges in using your classroom libraries to 
support instruction? If so, please describe. 
Oo How have or will these challenges be addressed? 


13. Does your school have Listening Centers (where students can access models of fluency and 


record themselves to assess their own fluency)? LlYes LINo (IF NO SKIP TO Q14) 


a. Where are these Listening Centers located? 


L] In the classrooms only 


L] Outside of the classrooms only (e.g., computer lab) 
oO How accessible are they? 


LJ Both 


b. Do you use the Listening Centers? LlYes LINo 


c. (If not used) Why not? 


d. (If used) How are you using them? 
fe) For which type of activities? (Probe: activities designed to access models of 
fluency; to assess students’ own fluency) 
O Are you using the Listening Centers with all students or subgroups of these 
students? 
fe) Please provide an example of how you use the Listening Centers to help 
differentiate instruction. 


e. (If used) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 


oO How were or will these challenges be addressed? 


14. Do you have Media Centers (3 computers and 1 printer) in your classrooms)? 


LlYes LINo (IF NO SKIP TO Q15) 
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a. Do you use the Media Centers? Ll Yes LINo 
b. (If not used) Why not? 


c. (If used) How are you using them? 
O For which type of activities? 
O Are you using the Media Centers with all students or subgroups of these students? 
O Please provide an example of how you use the Media Centers to help differentiate 
literacy instruction. 


d. (If used) Have you or your students encountered any challenges when using this 
technology? If so, please describe. 
oO How were or will these challenges be addressed? 
15. Do you and/or students in your school have access to Palm Pilots/Handheld Computers? 


LlYes L] No > If no, why not? (THEN SKIP TO Q16) 


a. Do you and/or your students use them in your classroom? Ll Yes LINo 


b. (If not used either by Teachers or by Students) Why not? 


c. (Ifused), how are they being used? 
oO For which type of activities do you use the Handheld Computers? 
o Are the Handheld Computers being used with all students or subgroups of 
students? 
oO Please provide an example of how you use the Handheld Computers to help 
differentiate literacy instruction. 


d. (If used) Have you or your students encountered any challenges when using this 
technology? If so, please describe. 
Oo How were or will these challenges be addressed? 
16. Overall, what are the strengths of the Striving Readers initiative? 
a. In your opinion, what factors do you anticipate will help to facilitate the successful 
implementation (or fidelity to the model) of Striving Readers during the current school year? 


At the school level? At the classroom level? 


17. Overall, what challenges have you encountered or do you anticipate in the current year to 
implementing the Striving Readers initiative? 


a. How might these challenges be addressed? 
18. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2010 School Visits, Wave II 
Self-Contained Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in identifying best practices 
and gaining a better understanding of how your work can be better supported. [I/another member of the 
evaluation team] had interviewed some of you and/or other classroom teachers about this last fall, and 
today I’d like to follow-up on that conversation. In particular, I’d like to focus on any changes in the 
implementation of the program at your school—overall and for each program component—that have 
occurred since the fall interview.I’d also like to touch upon any new successes since last fall, as well as 
the challenges and anticipated developments that you had identified during the fall interview. 


This focus group will take about 60 minutes. I would like to tape this [focus group/interview] to be sure I 
have recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are 
any specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to you or your school. Is this all right? 


Note to interviewer — How to conduct this interview: 


a) For all topics, probe first for new successes respondent has achieved since the fall interview, and 
request concrete examples of each. 


b) For all topics, probe also for the resolution of any challenges since the last interview and any new 
challenges that have arisen since the last interview, including any challenges, successes or 
changes in program implementation that may not have been identified or anticipated in the fall. 


c) Make sure to leave time for questions 19 and 20. 


1. Please introduce your[self/selves] and tell us how long have you been in this school, and which 
grade level(s) you teach this year. 


We would like to ask you about additional successes you have had in each area since the fall 
interview, as well as the status of any challenges or anticipated changes that you had mentioned last 
fall. Let’s discuss these successes and challenges as they relate to each of the topics that we 
discussed in the fall. 
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2. Specific Striving Readers instructional frameworks and materials 


e Instructional Frameworks(for example: whole part whole instruction, differentiating 
instruction, independent reading, small group instruction) 

e Core ComprehensionStrategies(Summarizing, Visualizing, Questioning, Metacognition, 
Inferring, Predicting, Text structure) 

e Core Comprehension Techniques (for example: Marzano’s vocabulary, Words their Way, 
PRC2, KWL, Word Study/word sorting, Read Aloud/Think Aloud, Reading Response, INSERT 
notes, ReQuest, List-Group-Label, etc.) 

e Striving Readers Instructional Materials(reading response notebooks, vocabulary notebooks, 
text sets distributed by the Striving Readers program (i.e., collections of short books centered 
around specific content area themes, written at a variety of reading levels so students can access 
the books independently) reading anthologies/basals, teacher text set guides, etc.) 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


3. Selecting appropriate student reading materials 


e use of fluency snapshots, interest inventories, identifying student reading levels; use of classroom 
libraries for self selected reading 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


4. Use of Grade-Level Team(s) to share knowledge and to plan and coordinate instruction and 
student groupings for differentiated instruction, which refers to providing to groups of students 
different content and/or instructional techniques specifically tailored to meet students’ 
individual educational needs and/or learning styles. 


Probe: ensure that grade level team is understood as a “horizontal” team consisting of staff across subject 
areas from the same grade. 

Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 

Probe: have any new challenges arisen since the fall interview. 


5. Use of a Literacy Team to share knowledge and to plan and coordinate instruction and student 
groupings for differentiated instruction 


Probe: ensure that literacy team is understood as a “vertical” team focusing on literacy issues across grade 
levels. 

Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 

Probe: have any new challenges arisen since the fall interview. 


6. Collaboration with the Literacy Intervention Teacher to provide blended instruction 
(scaffolding, differentiation for all students; addressing the needs of Tier 2 & 3 students) 
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Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


7. Collaboration with the District Coordinator to support implementation of Striving Readers 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


8. Using student data to plan small group activities, monitor success, and select appropriate 
reading materials 


e assessment data, demographic data, behavioral records, etc. 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


9. Comfort and successes/challenges integrating literacy as part of content area instruction, 
including specific Striving Readers instructional frameworks 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


10. Support from key experts for integrating literacy into content area instruction 


a. Literacy Intervention Teacher 
b. Literacy Coach 

c. Striving Readers Coordinator 
d. Regular ELA teachers 

e. Other staff member 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


11. Professional Development provided by Striving Readers to support integration of literacy into 
other content areas 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


12. Other kinds of support to help non-literacy staff integrate literacy into your content area 
instruction? 


Probe: new successes since the fall interview, including concrete examples 


Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
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13. Use of Striving Readers texts (i.e. sets of reading materials with differentiated structures and 
levels centered around specific content area themes, designed to improve student literacy in 
other subject area classes) 

Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 

Probe: have any new challenges arisen since the fall interview. 

14. Building and using classroom libraries to support instruction 

Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 

Probe: have any new challenges arisen since the fall interview. 

15. Using Listening Centers to support differentiated instruction 

Probe: new successes since the fall interview, including concrete examples 


Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


16. UsingMedia Centers to support differentiated instruction 

Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 

17. Using Palm Pilots/Handheld Computers to support differentiated instruction 
Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 

18. Additional Miscellaneous Challenges/Anticipated Changes from the Fall Interview 
Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


19. What factors do you anticipate will help to facilitate the successful implementation of Striving 
Readers in the 2010-2011 school year? 


20. What factors might hinder implementation next year? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Fall 2009 School Visits 
English Language Arts (ELA) Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you may know, your school was among six schools that 
were selected for an in-depth case study in part because they were identified by the district as being 
successful in their implementation of the Striving Readers initiative. We are interested in learning about 
the implementation of this program in your school, overall and for each program component, identifying 
best practices, and gaining a better understanding of the facilitating conditions and challenges to 
implementation. This interview is designed to gather information parallel to what was discussed in the 
prior two program years, but focusing on program plans and activities for the current school year. Since 
we are requesting a lot of information and I know you have a busy schedule, please be as succinct as you 
can in your answers. You will have an opportunity to elaborate further at the end of the focus group. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 
recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 
specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to yourself or your school. Is this all right? 


9. Please introduce yourselves and tell us how long have you been in this school, and which grade 
level(s) you teach this year. 


10. We would like to know about your use of specific Striving Readers instructional frameworks, 
techniques and strategies. 


a. How comfortable are you with the different frameworks, techniques and strategies identified 
in the Striving Readers grant? When did you start using them? 


Probe: 


o Striving Readers instructional frameworks (for example: whole part whole 
instruction, differentiating instruction, independent reading, small group 
instruction) 


o Core Comprehension Strategies (i.e., Summarizing, Visualizing, Questioning, 


Metacognition, Inferring, Predicting, Text structure) 
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o Core Comprehension Techniques (for example: Marzano’s vocabulary, PRC2, 
KWL, Word Study/word sorting, Read Aloud/Think Aloud, Reading Response, 
INSERT notes, PLAN, ReQuest, List-Group-Label, etc.) 


b. Which ones have you found to be the most successful and why? Which ones have presented 
the greatest challenges and why? Please provide examples. 
11. We would like to know more about the process that you, teachers and students use to select 
appropriate student reading materials. 


a. Do you use fluency snapshots and interest inventories to guide students’ selection of reading 
materials for their wide reading? Please provide examples. 


b. Are student reading levels used to help select appropriate materials? Please explain and 
provide examples. 


c. To what extent do students know how to select materials for wide reading that are appropriate 
to their abilities? Please explain and provide examples. 


Next, I would like to ask you a few questions about the specifics of when you meet with other 


teachers to plan and coordinate instruction and student groupings for the purpose of providing 
differentiated instruction that is appropriate to each student’s needs. 


12. Does your school have Grade-Level Teams? LlYes LINo (IF NO, SKIP TO QS) 


a. Are any of you involved in this team? LlyYes LINo (IF NO, SKIP TO Q5) 


b. What is your role in these teams? What topics are discussed in these meetings? 


13. Does your school have a Literacy Team? L]Yes LINo (IF NO, SKIP TO Q6) 
a. Are any of you involved in this team? L] Yes LINo (IF NO, SKIP TO Q6) 


b. What is your role in this team? What topics are discussed in these meetings? 
c. Is this team specific to the grades that Striving Readers serves (6" through 8")? 


14. We would like to know more about your collaboration with the Literacy Intervention Teacher 
and how you plan together to provide blended, differentiated instruction. 


a. In which setting(s) do you meet or collaborate with the LIT? 
O One-on-one meetings: 
Oo Grade-level team meetings: 
oO Literacy team meetings: 
oO Other: 


b. Do you collaborate with the LIT in scaffolding instruction (modeling strategies of 


comprehension, vocabulary and fluency while supporting steadily increasing levels of 
independence)? Please provide examples. 


A-139 


c. Do you plan together with the LIT for blended instruction that differentiates small group 
work? Please provide examples. 
0 Do you collaborate with the LIT to use assessment or other data to fine-tune 
differentiated instruction? Please provide examples. 


d. Do you plan together with the LIT to ensure that Tier 2 and 3 students receive appropriately 
targeted, differentiated instruction during classroom push-in sessions and AMP after-school 
lessons? Please provide examples. 


e. Have there been any changes in the strategies, activities and resources that you use during 
these push-in sessions, as compared to last year? 
oO If yes, please describe. 
Oo Why did these changes come about? 


f. What challenges, if any, have you encountered while working with the LIT and/or Tier 2 and 
3 students this year? 
Oo How were or will these challenges be addressed? 


15. We would like to learn more about your collaboration with the District Coordinator. 
a. Do you meet with him/her? If so, how often do you meet with him/her? 
___ Oncea ___ Several ___ Once ___ Several ____ Daily or 


month or less times a month a week times a week almost daily 


b. In what ways do you collaborate? What topics do you discuss? In what ways does he/she 
support your implementation of Striving Readers? 


c. Is there any additional support they could provide you with? 
16. In what other ways, on your own or in collaboration with the LIT, do you use student data? 
(This might include assessment data as well as other types of student data such as demographic 
and behavioral records.) 


a. Which kinds of data are you using? 


b. In which setting(s) (e.g., individually, as part of grade-level teams, as part of the Literacy 
Leadership Team, other) do you use these data? 


c. For what purposes? Please describe and provide examples. 


Probe & request examples: 

oO Plan for small group activities, 

O Monitor students' success in learning techniques for developing comprehension and using 
reading strategies appropriately, 

0 Select appropriate materials at students' independent and instructional levels. 
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We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 


17. One of the components of Striving Readers is building your classroom libraries. 
a. How do you use the classroom libraries? 


Probe & request examples: 
[1 For independent reading? 


For small group instruction? 


To support content area instruction? 


For read alouds? 


b. Do you use interest inventories to help students self select reading material and to guide your 
purchases? Please describe the process. 


c. How do you organize books in your classroom library? (Probe: Is there more than one 
criterion used to organize the libraries?) 


d. Have you or your students encountered any challenges in using your classroom libraries to 
support instruction? If so, please describe. 
0 How have or will these challenges be addressed? 


18. Does your school have Listening Centers (where students can access models of fluency and 


record themselves to assess their own fluency)? LlYes LINo (IF NO SKIP TO Q1 1) 
a. Where are these Listening Centers located? 


LJ In the classrooms only L] Outside of the classrooms only (e.g., computer lab) 
oO How accessible are they? 


LJ Both 


b. Do you use the Listening Centers? LlYes LINo 


c. (If not used) Why not? 


d. (If used) How are you using them? 
O For which type of activities? (Probe: activities designed to access models of 
fluency; to assess students’ own fluency) 
O Are you using the Listening Centers with all students or subgroups of these 
students? 
fe) Please provide an example of how you use the Listening Centers to help 
differentiate instruction. 
e. (If used) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 
Oo How were or will these challenges be addressed? 
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19. Do you have Media Centers (3 computers and 1 printer) in your classrooms)? 
Llyes LJNo (IF NO SKIP TO Q12) 


a. Do you use the Media Centers? LlYes LINo 


b. (If not used) Why not? 


c. (If used) How are you using them? 
O For which type of activities? 
O Are you using the Media Centers with all students or subgroups of these students? 
O Please provide an example of how you use the Media Centers to help differentiate 
instruction. 


d. (Ifused) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 
oO How were or will these challenges be addressed? 


20. Do you and/or students in your school have access to Palm Pilots/Handheld Computers? 
LlYes L] No > Ifno, why not? (THEN SKIP TO Q13) 


a. Do you and/or your students use them in your classroom? L] Yes LINo 


b. (if not used either by Teachers or by Students) Why not? 


c. (fused), how are they being used? 
oO For which type of activities do you use the Handheld Computers? 
o Are the Handheld Computers being used with all students or subgroups of 
students? 
oO Please provide an example of how you use the Handheld Computers to help 
differentiate instruction. 


d. (Ifused) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 
Oo How were or will these challenges be addressed? 
21. Overall, what are the strengths of the Striving Readers? 
a. In your opinion, what factors do you anticipate will help to facilitate the successful 
implementation (or also described as high fidelity to the model) of Striving Readers during 


the current school year? At the school level? At the classroom level? 


22. Overall, what challenges have you encountered or do you anticipate in the current year to 
implementing the Striving Readers initiative? 


a. How might these challenges be addressed? 


23. Is there anything else you would like to add regarding the literacy activities in your school? 
Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2010 School Visits, Wave II 
English Language Arts (ELA) Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in identifying best practices 
and gaining a better understanding of how your work can be better supported. [I/another member of the 
evaluation team] had interviewed some of you and/or other ELA teachers about this last fall, and today 
I'd like to follow-up on that conversation. In particular, I’d like to focus on any changes in the 
implementation of the program at your school—overall and for each program component—that have 
occurred since the fall interview.I’d also like to touch upon any new successes since last fall, as well as 
the challenges and anticipated developments that you had identified during the fall interview. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 
recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 
specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to you or your school. Is this all right? 


Note to interviewer — How to conduct this interview: 


a) For all topics, probe first for new successes respondent has achieved since the fall interview, and 
request concrete examples of each. 


b) For all topics, probe also for the resolution of any challenges since the last interview and any new 
challenges that have arisen since the last interview, including any challenges, successes or 
changes in program implementation that may not have been identified or anticipated in the fall. 


c) Make sure to leave time for questions 14 and 15. 


1. Please introduce your[self/selves] and tell us how long have you been in this school, and which 
grade level(s) you teach this year. 


We would like to ask you about additional successes you have had in each area since the fall 
interview, as well as the status of any challenges or anticipated changes that you had mentioned last 
fall. Let’s discuss these successes and challenges as they relate to each of the topics that we 
discussed in the fall. 
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2. Specific Striving Readers instructional frameworks and materials 


e Instructional Frameworks(for example: whole part whole instruction, differentiating 
instruction, independent reading, small group instruction) 

e Core Comprehension Strategies(Summarizing, Visualizing, Questioning, Metacognition, 
Inferring, Predicting, Text structure) 

e Core Comprehension Techniques (for example: Marzano’s vocabulary, Words their Way, 
PRC2, KWL, Word Study/word sorting, Read Aloud/Think Aloud, Reading Response, INSERT 
notes, ReQuest, List-Group-Label, etc.) 

e Striving Readers Instructional Materials (reading response notebooks, vocabulary notebooks, 
reading anthologies, reading basals, teacher text set guides (i.e., collections of short books 
centered around specific content area themes, written at a variety of reading levels so students can 
access the books independently), etc.) 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


3. Selecting appropriate student reading materials. 


e use of fluency snapshots, interest inventories, identifying student reading levels; use of classroom 
libraries for self selected reading 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


4. Use of Grade-Level Team(s)to plan and coordinate instruction and student groupings for 
differentiated instruction which refers to providing to groups of students different content 
and/or instructional techniques specifically tailored to meet students’ individual educational 
needs and/or learning styles. 


Probe: ensure that grade level team is understood as a “horizontal” team consisting of staff across subject 
areas from the same grade. 

Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 

Probe: have any new challenges arisen since the fall interview. 


5. Use of a Literacy Teamto share knowledge and to plan and coordinate instruction and student 
groupings for differentiated instruction. 


Probe: ensure that literacy team is understood as a “vertical” team focusing on literacy issues across grade 
levels. 

Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 

Probe: have any new challenges arisen since the fall interview. 


6. Collaboration with the Literacy Intervention Teacher to provide blended instruction 
(scaffolding, differentiation for all students; addressing the needs of Tier 2 - 3 students) 
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Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


7. Collaboration with the District Coordinator to support implementation of Striving Readers 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


8. Using student data to plan small group activities, monitor success, and select appropriate 
reading materials 


e assessment data, demographic data, behavioral records, etc. 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
9. Using your classroom libraries to support instruction 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
10. UsingListening Centersto support differentiated instruction 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
11. Using Media Centers (3 computers and 1 printer)to support differentiated instruction 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
12. Using Palm Pilots/Handheld Computers to support differentiated instruction 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
13. Additional Miscellaneous Challenges/Anticipated Changes from the Fall Interview 
Probe: new successes since the fall interview, including concrete examples 


Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
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14. What factors do you anticipate will help to facilitate the successful implementation of Striving 
Readers in the 2010-2011 school year? 


15. What factors might hinder implementation next year? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Fall 2009 School Visits 
Non-ELA Content Area Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you may know, your school was among six schools that 
were selected for an in-depth case study in part because they were identified by the district as being 
successful in their implementation of the Striving Readers initiative. We are interested in learning about 
the implementation of this program in your school, overall and for each program component, identifying 
best practices, and gaining a better understanding of the facilitating conditions and challenges to 
implementation. This interview is designed to gather information parallel to what was discussed in the 
prior two program years, but focusing on program plans and activities for the current school year. Since 
we are requesting a lot of information and I know you have a busy schedule, please be as succinct as you 
can in your answers. You will have an opportunity to elaborate further at the end of the focus group. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 
recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 
specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to yourself or your school. Is this all right? 


1. [Interviewer: please go around the room and ask participants to introduce themselves] Please 
introduce yourselves and tell us how long you have been a teacher in this school, and which 
subject area(s) and what grade level(s) you teach this year. 

We would like to know more about the types of support that you have received this year to help 

you integrate literacy into other content area instruction. 


2. Have you met with any of the following literacy experts? How often? What did you discuss with 
them? 
a. Literacy Intervention Teacher? 
b. Lead Literacy Teacher/Literacy Coach? 
c. Striving Readers School/District Coordinator? 
d. Striving Readers Technology Coordinators? 
e. English Language Arts teachers? 
f. Other staff member(s)? Please specify: 


3. Have you ever participated in any professional development conducted by the Striving Readers 


program related to integrating literacy into other content area instruction? L]Yes LINo 
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a. If yes, when/what year did you start participating in this type of training? 


b. Which trainings did you participate in? (Probe and request specific examples: site-based 
professional development, Summer Institute and follow-up Institutes, Saturday Seminars) 


c. Which trainings were most useful and why? Which trainings were least useful and why? 
d. What areas or topics would you like to receive additional support or training in? 


4. What other kinds of support are available to help non-literacy staff integrate literacy into your 
content area instruction? 


5. What other kinds of support would you need to help you integrate literacy into your content 
area instruction? 


The next few questions are about the extent and relative success of your efforts to integrate literacy 
as part of your content area instruction. 


6. When, if at all, did you start integrating literacy into your content area instruction? (Probe for: 
this year, last year, before Striving Readers began?) 


7. Have you been able to use any of the specific frameworks, techniques and strategies identified 
in the Striving Readers grant? 
Probe: 


o Striving Readers Instructional Frameworks (for example: whole part whole 


instruction, differentiating instruction, independent reading, small group 
instruction) 


o Core Comprehension Strategies (Summarizing, Visualizing, Questioning, 


Metacognition, Inferring, Predicting, Text structure) 

o Core Comprehension Techniques (for example: Marzano’s vocabulary, PRC2, 
KWL, Word Study/word sorting, Read Aloud/Think Aloud, Reading Response, 
INSERT notes, PLAN, ReQuest, List-Group-Label, etc.) 


a. Ifnot, how have you approached integrating literacy in your classroom? 


b. Ifyou have used specific Striving Readers frameworks, techniques and strategies, please 
provide examples. 


8. Does your school have Striving Readers text sets (i.e., sets of reading materials of different 
structures and levels, centered around specific content area themes, designed to improve 


student literacy in other subject area classes)? LlyYes LINo (IF NO SKIP TO Q9) 
a. Are the text sets being used in the content area classrooms? 


oO Social Studies L] Not Used—why? [LJ] Used—how? L] Don’t Know 
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Oo Science L] Not Used—why? [LJ Used—how? LJ Don’t Know 


O Mathematics L] Not Used—why? L] Used—how? L] Don’t Know 


9. One of the components of Striving Readers is building your classroom libraries. 


a. How do you use the classroom libraries? To what extent do they support your efforts to 
integrate literacy into your instruction in other content areas? 


Probe & request examples: 


For content area? 


For independent reading? 


For small group instruction? 


For read alouds? 


b. Do you use interest inventories to help students self select reading material and to guide your 
purchases? 


c. How do you organize books in your classroom library? (Probe: Is there more than one 
criterion used to organize the libraries?) 


10. To what extent have you used technology to help integrate literacy into instruction in other 
content areas? 


a. Please describe and provide examples. [Probe: media centers (classroom computers and 
printers), listening centers, hand held computers] 


b. Have there been any changes in the use of technology as compared to last year? 
c. Have you and/or the students encountered any challenges when using technology? If so, 
please describe. 


o How were or will these challenges be addressed? 


We would also like to know more about the factors facilitating or hindering the implementation of 
Striving Readers as it relates to the integration of literacy instruction into the other content areas. 


11. In your opinion, what factors do you anticipate will help to facilitate the successful integration 
of literacy instruction into the other content areas during the current school year? At the school 


level? At the classroom level? 


12. Overall, what challenges have you encountered or do you anticipate in the current year related 
to integrating literacy into the content areas? 


a. How might these challenges be addressed? 
13. Is there anything else you would like to add? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2010 School Visits, Wave II 
Non-ELA Content Area Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in identifying best practices 
and gaining a better understanding of how your work can be better supported. [I/another member of the 
evaluation team] had interviewed some of you and/or other subject area teachers about this last fall, and 
today I’d like to follow-up on that conversation. In particular, I’d like to focus on any changes in the 
implementation of the program at your school—overall and for each program component—that have 
occurred since the fall interview. I’d also like to touch upon any new successes since last fall, as well as 
the challenges and anticipated developments that you had identified during the fall interview. 


This focus group will take about 60 minutes. I would like to tape this [focus group/interview] to be sure I 
have recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are 
any specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to you or your school. Is this all right? 


Note to interviewer — How to conduct this interview: 


a) For all topics, probe first for new successes respondent has achieved since the fall interview, and 
request concrete examples of each. 


b) For all topics, probe also for the resolution of any challenges since the last interview and any new 
challenges that have arisen since the last interview, including any challenges, successes or 
changes in program implementation that may not have been identified or anticipated in the fall. 


c) Make sure to leave time for questions 12 and 13. 


11. Please introduce your[self/selves] and tell us how long you have been a teacher in this school, 
and which subject area(s) and what grade level(s) you teach this year. 


We would like to ask you about additional successes you have had in each area since the fall 
interview, as well as the status of any challenges or anticipated changes that you had mentioned last 
fall. Let’s discuss these successes and challenges as they relate to each of the topics that we 
discussed in the fall. 
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12. Support from key experts for integrating literacy into content area instruction 


a. Literacy Intervention Teacher 
b. Literacy Coach 

c. Striving Readers Coordinator 
d. Regular ELA teachers 

e. Other staff member 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


13. Professional Development provided by Striving Readers to support integration of literacy into 
other content areas 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


14. Other kinds of support to help non-literacy staff integrate literacy into your content area 
instruction 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


15. Integrating literacy as part of content area instruction, including specific Striving Readers 
instructional frameworks and materials 


oO. Instructional Frameworks(for example: whole part whole instruction, differentiating 
instruction, independent reading, small group instruction) 

Oo Core Comprehension Strategies(Summarizing, Visualizing, Questioning, Metacognition, 
Inferring, Predicting, Text structure) 

oO Core Comprehension Techniques (for example: Marzano’s vocabulary, Words their Way, 
PRC2, KWL, Word Study/word sorting, Read Aloud/Think Aloud, Reading Response, 
INSERT notes, ReQuest, List-Group-Label, etc.) 

f. Striving Readers Instructional Materials (reading response notebooks, vocabulary 
notebooks, text sets (i.e., collections of short books centered around specific content area 
themes, written at a variety of reading levels so students can access the books independently), 
teacher text set guides, etc.) 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


16. Use of Striving Readers text sets (i.e. sets of reading materials with differentiated structures and 
levels centered around specific content area themes, designed to improve student literacy in 
other subject area classes) 


Probe: new successes since the fall interview, including concrete examples 


Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
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17. Building and using classroom libraries to support literacy instruction in your content area 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
11. Use of technology to support integration of literacy into content area instruction 
a. Listening Centers 
b. Media Centers (3 computers and 1 printer) 
c. Palm Pilots/Handheld Computers 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
12. Additional Miscellaneous Challenges/Anticipated Changes from the Fall Interview 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


13. What factors do you anticipate will help to facilitate the successful integration of literacy into 
the content areas in the 2010-2011 school year? 


14. What factors might hinder integration of literacy into the content areas next year? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Fall 2009 School Visits 
Literacy Intervention Teacher (LIT) Interview Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in learning about the 
implementation of this program in your school, overall and for each program component, identifying best 
practices, and gaining a better understanding of the facilitating conditions and challenges to 
implementation. This interview is designed to gather information parallel to what was discussed in the 
prior two program years, but focusing on program plans and activities for the current school year. Since 
we are requesting a lot of information and we have a limited period of time, please be as succinct as you 
can in your answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. The recordings will not be shared with anyone outside of Metis; however, your comments 
may be used in reporting findings from this case study. If there are any specific comments that you would 
like to keep confidential, let us know and we will make sure they cannot be tied to yourself or your 
school. Is this all right? 


24. How long have you been an LIT? If you were in this role last year, has your role as the Literacy 
Intervention Teacher (LIT) in the Striving Readers Initiative changed from last year? 
a. Ifso, how? 
25. We would like to learn more about your collaboration with the District Coordinator. 
a. How often do you meet with him/her? 
___Oncea ___ Several ____ Once ___ Several ___ Daily or 


month or less times a month a week times a week almost daily 


b. What topics do you discuss? In what ways does he/she support your implementation of 
Striving Readers? 


c. Is there any additional support they could provide you with? 


Next, I would like to ask you a few questions about the specifics of when you meet with classroom 
teachers to plan, prioritize and coordinate instruction, responsibilities, and student groupings. 


26. Do you have one-on-one meetings with the teachers outside of instruction time? 
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LlYes LINo (IF NO SKIP TO Q4) 


a. When do you have these meetings? 
____ Oncea ____ Several ____ Once ___ Several ____ Daily or 
month or less times a month a week times a week almost daily 
b. Please describe how these meetings are used to facilitate the differentiation of instruction for 
Tier 2 and 3 students during the classroom targeted intervention and/or after school classes. 


c. What topics are discussed in these meetings? 


d. Do you discuss assessment data or other types of student data (such as demographic data or 
behavioral records) in these meetings? If so, for what purposes? 


27. Does your school have Grade-Level Teams? LlYes LINo (IF NO, SKIP TO QS) 


c. Are you involved in these teams? LlYes LINo (IF NO, SKIP TO Q5) 


d. Please describe how these meetings are used to facilitate the differentiation of instruction for 
Tier 2 and 3 students during the classroom targeted intervention and/or after school classes (if 
at all). [If applicable]: How does this function differ from that of the one-on-one meetings? 


e. Do these teams use assessment data, or other types of student data (such as demographic data 
or behavioral records)? If so, for what purposes? [If applicable]: How does this differ from 
the use of data during one-on-one meetings? 


28. Does your school have a Literacy Team? L1]Yes LINo (IF NO, SKIP TO Q6) 
a. Are you involved in this team? LlyYes LINo (IF NO, SKIP TO Q6) 


b. Please describe how these meetings are used to ensure that Tier 2 and 3 students receive 
appropriately targeted, differentiated instruction during the classroom targeted intervention 
and/or after school classes. [If applicable]: How does this function differ from those of the 
one-on-one and Grade-Level Team meetings? 


c. Do these teams use assessment data, or other types of student data (such as demographic data 
or behavioral records)? If so, for what purposes? [If applicable]: How does this differ from 
the use of data during one-on-one and Grade-Level Team meetings? 


29. Have you encountered any challenges in collaborating with classroom teachers one-on-one or as 
part of the team meetings? 


If so, please describe the challenges and discuss how they were or will be addressed. 


30. I would like to know more about the targeted intervention and the work that you do with Tier 2 
and Tier 3 students during the regular school day. 


a. Which grades do you work with? How many classes? For how long (e.g., length of targeted 
instruction, number of periods per week per class)? 
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On average, how many kids are there in the targeted group per class? 


How do you determine the needs of your students that might impact on their literacy 
development? 
How do you develop appropriate instruction for them? 


What types of strategies, activities and resources do you use when you meet with your Tier 2 
& 3 students? 
O What types of student grouping do you use? 
oO What strategies do you use to differentiate instruction for students of different 
ability levels within this group? 


Have you encountered any challenges when working in the language arts classroom with Tier 
2 and 3 students and/or their teachers this year? If so, please describe. 

Oo How were or will these challenges be addressed? 
Have there been any changes in the strategies, activities and resources that you use during 
these targeted intervention sessions, as compared to last year? 

oO If yes, please describe. 

Oo Why did these changes come about? 


31. I would like to know more about the intensive intervention and the work that you do with Tier 
3 students during the after school program. 


a. 


b. 


What is your role in the after-school component of Striving Readers? 


Please describe the structure and content of the after-school Achieving Maximum Potential 
(AMP) programming for struggling readers. 


Are you using the AMP intervention software during the after-school program? 


LlYes LINo 
oO (Ifnot used) Why not? 


oO (If used) How is it being used? 
1. For which type of activities? 
2. Are you using it with all students or subgroups of students? 


oO (Ifused) Have you, students, and/or teachers encountered any challenges when 
using this technology? If so, please describe. 
" How were or will these challenges be addressed? 


oO (Ifused) In what ways, if any, has this technology improved instruction and student 
learning in language arts? 
What successes has the school had with the after school component of Striving Readers? 


What challenges has the school encountered with the after school component of Striving 
Readers? 


oO How were or will these challenges be addressed? 


Have there been any changes in the strategies, activities and resources that you use during the 
afterschool program, as compared to last year? 
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oO If yes, please describe. 

Oo Why did these changes come about? 
We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 


32. Do your teachers utilize the professional library (a collection of professional development 
resources on the topic of classroom-based literacy instruction provided through Striving 
Readers) in the school? If so, how? 


33. In what ways are the classroom libraries being used by the teachers? For what types of 
activities? 
a. What is your role, if any, in helping teachers use their classroom libraries? 


34. Does your school have Listening Centers (where students can access models of fluency and 


record themselves to assess their own fluency)? LlYes LINo (IF NO SKIP TO Q12) 


a. Do you use the Listening Centers to support your role in providing differentiated instruction 


to struggling readers? LlYes LINo 


b. (Uf not used) Why not? 


c. (fused) How are you using them? 
O For which type of activities? (Probe: activities designed to access models of 
fluency; to assess students’ own fluency) 
oO Are you using the Listening Centers with all struggling students or subgroups of 
these students? 
fe) Please provide an example of how you use the Listening Centers to help 
differentiate instruction. 


d. (If used) Have you, students, and/or teachers encountered any challenges when using this 
technology? If so, please describe. 
Oo How were or will these challenges be addressed? 
35. Does your school have Media Centers (3 computers and 1 printer in the classroom)? 


LlYes LINo (IF NO SKIP TO Q13) 


a. Do you use the Media Centers to support your role in providing differentiated instruction to 


struggling readers? LlYes LINo 


b. (If not used) Why not? 


c. (If used) How are you using them? 
fe) For which type of activities? 
O Are you using the Media Centers with all struggling students or subgroups of these 
students? 
O __— Please provide an example of how you use the Media Centers to help differentiate 
instruction. 
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d. Have you, students, and/or teachers encountered any challenges when using this technology? 
If so, please describe. 
Oo How were or will these challenges be addressed? 


36. Do you and/or students in your school have access to Palm Pilots/Handheld Computers? 
LlyYes L] No > If no, why not? (THEN SKIP TO Q14) 


a. Do you use the Handheld Computers to support your role in providing differentiated 


instruction to struggling readers? L1]Yes LINo 


b. Do your students use the Palm Pilots/Handheld Computers? L] Yes LINo 


(If not used either by LIT or by Students) Why not? 


ce 


d. (Ifused) Are they being used during the afterschool program? L] Yes LINo 
oO. If yes, how are they being used? 
1. For which type of activities do you use the Handheld Computers? 
2. Are the Handheld Computers being used with all struggling readers or 
subgroups of students? 
3. Please provide an example of how you use the Handheld Computers to help 
differentiate instruction. 


e. (Ifused) Are they being used as part of the targeted intervention during the regular school 
day? 
LlYes LINo 


o If yes, how are they being used? 
"For which type of activities do you use the Handheld Computers? 
= Are the Handheld Computers being used with all struggling readers or 
subgroups of students? 
" Please provide an example of how you use the Handheld Computers to help 
differentiate instruction. 


f. Have you, students, and/or teachers encountered any challenges when using this technology? 
If so, please describe. 
Oo How were or will these challenges be addressed? 


37. Overall, what are the strengths of your school’s literacy curriculum? 
a. In your opinion, what factors do you anticipate will help to facilitate the implementation of 
Striving Readers as defined by the model during the current school year? At the school level? 
At the classroom level? 
38. Overall, what challenges have you encountered or do you anticipate in the coming year to 
implementing the Striving Readers initiative? 
a. How might these challenges be addressed? 


39. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2010 School Visits, Wave II 
Literacy Intervention Teacher (LIT) Interview Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in identifying best practices 
and gaining a better understanding of how your work can be better supported. [I/another member of the 
evaluation team] had interviewed you about this last fall, and today I’d like to follow-up on that 
conversation. In particular, I’d like to focus on any changes in the implementation of the program at your 
school—overall and for each program component—that have occurred since the fall interview. I’d also 
like to touch upon any new successes since last fall, as well as the challenges and anticipated 
developments that you had identified during the fall interview. 


This interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. The recordings will not be shared with anyone outside of Metis. If there are any specific 
comments that you would like to keep confidential, let us know and we will make sure they cannot be tied 
to you or your school. Is this all right? 


Note to interviewer — How to conduct this interview: 


a) For all topics, probe first for new successes respondent has achieved since the fall interview, and 
request concrete examples of each. 


b) For all topics, probe also for the resolution of any challenges since the last interview and any new 
challenges that have arisen since the last interview, including any challenges, successes or 


changes in program implementation that may not have been identified or anticipated in the fall. 


c) Make sure to leave time for questions 14 and 15. 


We would like to ask you about additional successes you have had in each area since the fall 
interview, as well as the status of any challenges or anticipated changes that you had mentioned last 
fall. Let’s discuss these successes and challenges as they relate to each of the topics that we 
discussed in the fall. 


1. Collaboration with the District Coordinator to support the implementation of Striving Readers 
Probe: new successes since the fall interview, including concrete examples 


Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
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2. One-on-one collaboration with teachers outside of instruction time to plan how to differentiate 
instruction (which refers to providing to groups of students different content and/or 
instructional techniques specifically tailored to meet students’ individual educational needs 
and/or learning styles) during classroom targeted interventions or to discuss 
assessment/student data 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


3. Use of Grade-Level Team(s) to plan and coordinate instruction and student groupings for 
differentiated instruction or to discuss assessment/student data 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


4. Use of a Literacy Team to share knowledge and to plan and coordinate instruction and student 
groupings for differentiated instruction or to discuss assessment/student data 


Probe: ensure that literacy team is understood as a “vertical” team focusing on literacy issues across grade 
levels. 

Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 

Probe: have any new challenges arisen since the fall interview. 


5. Collaborating with classroom teachers one-on-one or as part of the team meetings 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


6. Targeted Intervention and work done with Tier 2 and Tier 3 students during regular school 
day — identifying and meeting the needs of struggling readers 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


7. The intensive intervention and your work with Tier 3 students during the after school 
program (AMP) 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 


8. Your teachers’ use of the school’s professional library 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
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Probe: have any new challenges arisen since the fall interview. 
9. Your teacher’s use of classroom libraries to support instruction? 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
10. Using Listening Centers to support differentiated instruction 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


11. Using Media Centers (3 computers and 1 printer in the classroom) to support differentiated 
instruction 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
12. Using Palm Pilots/Handheld Computers to support differentiated instruction? 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
13. Additional Miscellaneous Challenges/Anticipated Changes from the Fall Interview 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


14. What factors do you anticipate will help to facilitate the successful implementation of Striving 
Readers in the 2010-2011 school year? 


15. What factors might hinder implementation next year? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Fall 2009 School Visits 
Principal Interview Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in learning about the 
implementation of this program in your school, overall and for each program component, identifying best 
practices, and gaining a better understanding of the facilitating conditions and challenges to 
implementation. This interview is designed to gather information parallel to what was discussed in the 
prior two program years, but focusing on program plans and activities for the current school year. Since 
we are requesting a lot of information and I know you have a busy schedule, please be as succinct as you 
can in your answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. The recordings will not be shared with anyone outside of Metis; however, your comments 
may be used in reporting findings from this case study. If there are any specific comments that you would 
like to keep confidential, let us know and we will make sure they cannot be tied to you or your school. Is 
this all right? 


40. How long have you been the principal in this building? [If more than one year] Has your role 
in the Striving Readers Initiative changed from last year? 
a. Ifso, how? 


41. Does your school have a Literacy Team? L1]Yes LINo (IF NO, SKIP TO Q3) 


a. Which of your staff are members of the Literacy Team? 


LI] Principal LIAP or other Senior Administrator L]Grade level teacher(s) 
L Literacy Intervention Teacher [J]Lead Literacy Teacher LIELL/ESL Teacher(s) 
L]Librarian(s) LJ Special education teacher(s) LJ Other: 


b. How often does the Literacy Team meet? 


L] Has not met L] Less than once per month [J Once per month 
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L] Biweekly L] Weekly L] Several times a week or more 


c. What role does the Literacy Team play at your school? 
d. What role do you play in the Literacy Team? 


e. How does the team address the needs of struggling readers? 


42. Does your school have Grade-Level Teams? LlYes LINo (IF NO, SKIP TO Q4) 


a. Which of your staff are members of the Grade-Level Teams? 
b. How are the Grade-Level Teams structured? (single grades, clustered) 


c. How often do these teams meet? 


L] Have not met L] Less than once per month LJ Once per month 


L] Biweekly L] Weekly L] Several times a week or more 


d. What role do these teams play at your school? 


e. How do these teams address the needs of struggling readers? 


We would like to learn more about the use of assessment data and how that impacts instruction. 


43. In what ways, if any, is your school using assessment data beyond mandated reporting to the 
district and state? 


a. Please describe and provide examples, including distinctions between how different teams or 
individuals use the data. 
oO Student placement in specific groups, programs, etc. 
o Differentiate instruction 
oO Planning professional development 
o Other 


b. By whom? 

0 Principal/APs 
Literacy Team 
Grade-Level Teams 
LIT 
Other 


oo0°0 


44. Are any other types of data (e.g., demographic, behavioral) being used? 


a. By whom? 
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b. 


For what purposes? Please describe and provide examples. 


45. The AMP after school program is part of Striving Readers. We want to know more about your 
school’s use of this program. 


a. 


e. 


Do you feel that the AMP program is appropriate to the reading levels of the students who are 
currently participating? Are there any students in AMP who you feel should not be there? 
(Reading levels too high/too low?) Are there students who are not in AMP who should be? 


Are the LIT or other after-school teachers using the AMP activities and materials? Are they 
using any supplemental materials or strategies? 


Are students using the AMP software? LlYes LINo 


o Ifnot, why not? 


What successes has the school had with implementing the AMP program strategies, including 
the use of the AMP software? 


What challenges has the school encountered with the AMP program and software? 


We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 


46. To what extent has the use of technology been integrated into literacy instruction? 


a. 


Please describe and provide examples. [Probe: media centers (classroom computers and 
printers), listening centers, hand held computers] 


Have there been any changes in the use of technology as compared to last year? 


Have the students and/or teachers encountered any challenges when using technology? If so, 
please describe. 
o How were or will these challenges be addressed? 
O Have you or your teachers had any interaction with one or both of the Striving 
Readers Technology Coordinator? If so, please describe. 


In what ways, if any, has this technology improved instruction and student learning in 
language arts? 
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47. Now I would like to ask about your school’s efforts, if any, to integrate literacy into the content 
areas. 
a. Please describe your school’s efforts, if any, to integrate literacy into the content areas. 


b. Does your school have text sets (i.e., sets of reading materials of different structures and 
levels, centered around specific content area themes, designed to improve student literacy in 


other subject area classes) provided through the Striving Readers program? LlYes LJNo 
oO If yes, are the text sets being used in the content area classrooms? 


Social Studies LJ] Not Used—why? [J Used—how? LJ] Don’t Know 
Science L] Not Used—why? LJ] Used—how? L] Don’t Know 
Mathematics L] Not Used—why? LL] Used—how? L] Don’t Know 


c. Do non-literacy staff participate in professional development for the Striving Readers 


project? (Includes: Bilingual, SPED, Math, Science, Social Studies teachers) LIYes LINo 


o If yes, who has received professional development? Please specify staff positions. 
oO What topics were covered? 


d. What other kinds of support are there to help non-literacy staff integrate literacy into their 
content areas? 


e. What are some of the challenges that these teachers are facing? 
Oo How might these challenges be addressed? 


Now we would like to know more about different types of support that your school has received for 
the implementation of Striving Readers. 


48. Please describe your school’s partnership with National-Louis University and Donna Ogle. 
a. In what ways has her support been helpful to the implementation of Striving Readers? 
b. Is there any additional support she could provide you with? 


49. What type of support has the District Coordinator provided to you and your school? Please 
describe. 


a. To what extent has his or her support been helpful for the implementation of Striving 
Readers? Please explain. 


b. Is there any additional support they could provide you with? 


50. Has your role as instructional leader changed as a result of your participation in Striving 
Readers? If so, how? 
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51. Overall, what are the strengths of your school’s literacy curriculum? 
a. In your opinion, what factors do you anticipate will help to facilitate the implementation of 
Striving Readers as defined by the model during the current school year? At the school level? 


At the classroom level? 


52. Overall, what challenges have you encountered or do you anticipate in the coming year to 
implementing the Striving Readers initiative? 


a. How might these challenges be addressed? 
53. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2010 School Visits, Wave II 
Principal Interview Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in identifying best practices 
and gaining a better understanding of how your work can be better supported. [I/another member of the 
evaluation team] had interviewed you about this last fall, and today I’d like to follow-up on that 
conversation. In particular, I’d like to focus on any changes in the implementation of the program at your 
school—overall and for each program component—that have occurred since the fall interview. I’d also 
like to touch upon any new successes since last fall, as well as the challenges and anticipated 
developments that you had identified during the fall interview. 


This interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. The recordings will not be shared with anyone outside of Metis. If there are any specific 
comments that you would like to keep confidential, let us know and we will make sure they cannot be tied 
to you or your school. Is this all right? 


Note to interviewer — How to conduct this interview: 


a) For all topics, probe first for new successes respondent has achieved since the fall interview, and 
request concrete examples of each. 


b) For all topics, probe also for the resolution of any challenges since the last interview and any new 
challenges that have arisen since the last interview, including any challenges, successes or 


changes in program implementation that may not have been identified or anticipated in the fall. 


c) Make sure to leave time for questions 11 and 12. 


We would like to ask you about additional successes you have had in each area since the fall 
interview, as well as the status of any challenges or anticipated changes that you had mentioned last 
fall. Let’s discuss these successes and challenges as they relate to each of the topics that we 
discussed in the fall. 
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1. Use of a Literacy Team to share knowledge and to plan and coordinate instruction and student 
groupings for differentiated instruction which refers to providing to groups of students 
different content and/or instructional techniques specifically tailored to meet students’ 
individual educational needs and/or learning styles. 


Probe: ensure that literacy team is understood as a “vertical” team focusing on literacy issues across grade 
levels. 

Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 

Probe: have any new challenges arisen since the fall interview. 


2. Use of Grade-Level Team(s) to plan and coordinate instruction and student groupings for 
differentiated instruction. 


Probe: ensure that grade level team is understood as a “horizontal” team consisting of staff across subject 
areas from the same grade. 

Probe: new successes since the fall interview, including concrete examples 

Probe: have any of the challenges faced during the fall been resolved during the year 

Probe: have any new challenges arisen since the fall interview. 


3. Use of assessment data to inform literacy instruction or plan professional development 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
4. Use of other student data (e.g., demographic, behavioral) to inform literacy instruction 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
5. Use of the AMP after school program to support literacy instruction at your school 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
6. Use of technology to support literacy instruction 
e Listening Centers 
e Media Centers (3 computers and 1 printer) 
e Palm Pilots/Handheld Computers 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


7. Integration of literacy instruction into content areas 
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e Use of Striving Readers text sets(i.e., collections of short books centered around specific 
content area themes, written at a variety of reading levels so students can access the 
books independently) 

e professional development for non-literacy staff 


Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
8. Support provided by National-LouisUniversity and Donna Ogle. 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 
9. Support provided by Striving Readers District Coordinator 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 
Probe: have any new challenges arisen since the fall interview. 

10. Additional Miscellaneous Challenges/Anticipated Changes from the Fall Interview 
Probe: new successes since the fall interview, including concrete examples 
Probe: have any of the challenges faced during the fall been resolved during the year 


Probe: have any new challenges arisen since the fall interview. 


11. What factors do you anticipate will help to facilitate the successful integration of literacy into 
the content areas in the 2010-2011 school year? 


12. What factors might hinder implementation next year? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2010 School Visits [New Schools] 
Self-Contained Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. Your school was among six schools that were selected for an 
in-depth case study in part because they were identified by the district as being successful in their 
implementation of the Striving Readers initiative. We are interested in learning about the implementation 
of this program in your school, overall and for each program component, identifying best practices, and 
gaining a better understanding of the facilitating conditions and challenges to implementation. Since we 
are requesting a lot of information and I know you have a busy schedule, please be as succinct as you can 
in your answers. You will have an opportunity to elaborate further at the end of the focus group. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 
recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 
specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to yourself or your school. Is this all right? 


1. Please introduce yourselves and tell us how long have you been in this school, and which grade 
level(s) you teach this year. 


2. We would like to know about your use of specific Striving Readers instructional 
frameworks, techniques and strategies. 


a. How comfortable are you with the different frameworks, techniques and strategies identified 
in the Striving Readers grant? When did you start using them? 


b. Which oneshave you found to be the most successful and why? Which ones have presented 
the greatest challenges and why? 


Probe and request examples: 


o Striving Readersinstructional frameworks(for example: whole part whole 
instruction, differentiating instruction, independent reading, small group 
instruction) 


Probe: If respondents mention differentiated instruction, probe for difference 
between true differentiation and small group instruction, i.e., providing different 
content and/or instructional techniques specifically tailored to meet students’ 
individual educational needs and/or learning styles 
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o Core Comprehension Strategies(Summarizing, Visualizing, Questioning, 


Metacognition, Inferring, Predicting, Text structure) 

o Core Comprehension Techniques (for example: Marzano’s vocabulary, Words 
their Way, PRC2, KWL, Word Study/word sorting, Read Aloud/Think Aloud, 
Reading Response, INSERT notes, PLAN, ReQuest, List-Group-Label, etc.) 


3. We would like to know more about the process that you, teachers and students use to select 
appropriate student reading materials. 


a. Do you use fluency snapshots and interest inventories to guide students’ selection of reading 
materials for their wide reading? Please provide examples. 


b. Are student reading levels used to help select appropriate materials? Please explain and 
provide examples. 


c. To what extent do students know how to select materials for wide reading that are appropriate 
to their abilities? Please explain and provide examples. 


Next, I would like to ask you a few questions about the specifics of when you meet with other 
teachers to plan and coordinate instruction and student groupings for the purpose of providing 


differentiated instruction that is appropriate to each student’s needs. 


4. Does your school have Grade-Level Teams (“horizontal” teams consisting of staff across subject 
areas from the same grade)? (Yes (No (IF NO, SKIP TO Q5) 


f. Are you all involved in these teams? 
g. What is your role in these teams? What topics are discussed in these meetings? 


5. Does your school have a Literacy Team (a “vertical” team focusing on literacy issues across 
grade levels)? “Yes (CINo (IF NO, SKIP TO Q6) 


d. Are any of you involved in this team? (“Yes (No (IF NO, SKIP TO Q6) 
e. What is your role in this team? What topics are discussed in these meetings? 


6. We would like toknow more about your collaboration with the Literacy Intervention Teacher 
and how you plan together to provide blended, differentiated instruction. 


a. In which setting(s) do you meet or collaborate with the LIT? 
O One-on-one meetings: 
oO Grade-level team meetings: 
oO Literacy team meetings: 


b. Do you collaborate with the LIT in scaffolding instruction (comprehension, vocabulary and 
fluency)? Please provide examples. 
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c. Do you plan together with the LIT for blended instruction that differentiates small group 
work? Please provide examples. 

0 Do you collaborate with the LIT to use assessment or other data to fine-tune 
differentiated instruction? Please provide examples. (Interviewer: be sure 
respondent understands ‘differentation’ to mean providing different content and/or 
instructional techniques specifically tailored to meet students’ individual 
educational needs and/or learning styles) 


d. Please describe other topics you discuss when meeting with the LIT to ensure that Tier 2 and 
3 students receive appropriately targeted, differentiated instruction during in-class targeted 
intervention sessions and AMP after-school lessons. (Interviewer: be sure respondent 
understands ‘differentation’ to mean providing different content and/or instructional 
techniques specifically tailored to meet students’ individual educational needs and/or 
learning styles) 


e. Have there been any changes in the strategies, activities and resources that you use during 
these push-in sessions, as compared to last year? 
oO If yes, please describe. 
oO Why did these changes come about? 


f. What challenges, if any, have you encountered while working with the LIT and/or Tier 2 and 
3 students this year? 
Oo How were or will these challenges be addressed? 


7. We would like to learn more about your collaboration with the District Coordinator. 
a. Do you meet with him/her? If so, how often do you meet with him/her? 
___Oncea ___ Several ___ Once ___ Several ____ Daily or 


month or less times a month a week times a week almost daily 


b. In what ways do you collaborate? What topics do you discuss? In what ways does he/she 
support your implementation of Striving Readers? 


c. Is there any additional support they could provide you with? 
8. In what other ways, on your own or in collaboration with the LIT, do you use student data? 
(This might include assessment data as well as other types of student data such as demographic 
and behavioral records.) 


a. Which kinds of data are you using? 


b. In which setting(s) (e.g., individually, as part of grade-level teams, as part of the Literacy 
Team, other) 


c. For what purposes? Please describe and provide examples. 


Probe & request examples: 

oO Plan for small group activities, 

O Monitor students' success in learning techniques for developing comprehension and using 
reading strategies appropriately, 

0 Select appropriate materials at students' independent and instructional levels. 
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We would like to know more about your efforts, if any, to integrate literacy into other content area 
instruction. 


9. When, if at all,did you start integrating literacy into your content area instruction? (Probe for: 
this year, last year, before Striving Readers began?) 


10. Have you been able to use any of the specific frameworks, techniques and strategies identified 
in the Striving Readers grant? 


a. Ifnot, how have you approached integrating literacy in your classroom? 


b. Ifyou have used specific Striving Readers frameworks, techniques and strategies, please 
provide examples. 


11. We would also like to know more about the types of support that you have received to help you 
integrate literacy instruction into other content areas. 


a. Have you met with any literacy experts to specifically discuss the integration of literacy into 
other content area instruction? How often? What did you discuss with them? [Probe for: 
Literacy Intervention Teacher, Literacy Coach, Striving Readers Coordinator, other] 


b. Have you ever participated in any professional development conducted by the Striving 
Readers program related to integrating literacy into other content area instruction? Yes 
LINo 


0 If yes, when/what year did you start participating in this type of trainings? 

oO Which trainings did you participate in? (Probe and request specific examples: site-based 
professional development, Summer Institute and follow-up Institutes, Saturday Seminars) 

oO Which trainings were most useful and why? Which trainings were least useful and why? 

O What areas or topics would you like to receive additional support or training in? 


c. Does your school have text sets developed and distributed by the Striving Readers program 
(i.e., collections of short books centered around specific content area themes, written at a 
variety of reading levels so students can access the books independently) with accompanying 
teacher guides? Yes 0 No 


If yes, are the Striving Readers text sets being used in the content area classrooms? 
oO Social Studies co Not Used—why? o Used—how? 0 Don’t Know 
oO Science 0 Not Used—why? o Used—how? o Don’t Know 


d. What other kinds of support areavailable to help you integrate literacy into your content area 
instruction? 


e. What other kinds of support would you need to help you integrate literacy into your content 
area instruction? 
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We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 


12. One of the components of Striving Readers is building your classroom libraries. 
a. How do you use the classroom libraries? 


Probe & request examples: 
[1 For independent reading? 


For small group instruction? 


To support content area instruction? 


For read alouds? 


b. Do you use interest inventories to help students self select reading material and to guide your 
purchases? Please describe the process. 


c. How do you organize books in your classroom library? (Probe: Is there more than one 
criterion used to organize the libraries?) 


d. Have you or your students encountered any challenges in using your classroom libraries to 
support instruction? If so, please describe. 


Oo How have or will these challenges be addressed? 


13. Does your school have Listening Centers (where students can access models of fluency and 
record themselves to assess their own fluency)? [Yes (No (IF NO SKIP TO Q14) 


f. Where are these Listening Centers located? 
(In the classrooms only 
Outside of the classrooms only (e.g., computer lab) 
oO How accessible are they? 
LJ] Both 


g. Do you use the Listening Centers? [Yes [INo 
h. (If not used) Why not? 


i. (fused) How are you using them? 
O For which type of activities? 
O Are you using the Listening Centers with all students or subgroups of these 
students? 
fe) Please provide an example of how you use the Listening Centers to help 
differentiate instruction. 


Probe for evidence of true differentiation, i.e., providing different content and/or 


instructional techniques specifically tailored to meet students’ individual 
educational needs and/or learning styles 
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j. (fused) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 


oO How were or will these challenges be addressed? 


14. Do you have Media Centers (3 computers and 1 printer) in your classrooms)? (|Yes_ (INo (IF 
NO SKIP TO Q15) 


e. Do you use the Media Centers? (Yes UNo 
f. (If not used) Why not? 


g. (If used) How are you using them? 
O For which type of activities? 
O Are you using the Media Centers with all students or subgroups of these students? 


O Please provide an example of how you use the Media Centers to help differentiate 
instruction. 


Probe for evidence of true differentiationi.e., providing different content and/or 
instructional techniques specifically tailored to meet students’ individual 
educational needs and/or learning styles 


h. (If used) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 


o How were or will these challenges be addressed? 


15. Do you and/or students in your school have access to Palm Pilots/Handheld Computers? 
(Yes (No? Ifno, why not? (THEN SKIP TO Q16) 


e. Do you and/or your students use them in your classroom? [1Yes LINo 
f. (Ifnot used either by Teachers or by Students) Why not? 


g. (if bas how are they being used? 
For which type of activities do you use the Handheld Computers? 
o Are the Handheld Computers being used with all students or subgroups of 
students? 


O Please provide an example of how you use the Handheld Computers to help 
differentiate instruction. 


Probe for evidence of true differentiationi.e., providing different content and/or 
instructional techniques specifically tailored to meet students’ individual 
educational needs and/or learning styles 


h. (If used) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 
Oo How were or will these challenges be addressed? 
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16. Overall, what are the strengths of the Striving Readers? 


a. In your opinion, what factors are helping to facilitate the implementation of Striving Readers 
as defined by the model during this school year? At the school level? At the classroom level? 


17. Overall, what challenges to implementing the Striving Readers initiative have you encountered 
during this school year? 


b. How might these challenges be addressed? 
18. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2010 School Visits [New Schools] 
English Language Arts (ELA) Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. Your school was among six schools that were selected for an 
in-depth case study in part because they were identified by the district as being successful in their 
implementation of the Striving Readers initiative. We are interested in learning about the implementation 
of this program in your school, overall and for each program component, identifying best practices, and 
gaining a better understanding of the facilitating conditions and challenges to implementation. Since we 
are requesting a lot of information and I know you have a busy schedule, please be as succinct as you can 
in your answers. You will have an opportunity to elaborate further at the end of the focus group. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 
recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 
specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to yourself or your school. Is this all right? 


1. Please introduce yourselves and tell us how long have you been in this school, and which grade 
level(s) you teach this year. 


2. We would like to know about your use of specificStriving Readers instructional 
frameworks, techniques and strategies. 


a. How comfortable are you with the different frameworks, techniques and strategies identified 
in the Striving Readers grant? When did you start using them? 


b. Which oneshave you found to be the most successful and why? Which ones have presented 
the greatest challenges and why? 


Probe and request examples: 


o Striving Readersinstructional frameworks (for example: whole part whole 


instruction, differentiating instruction, independent reading, small group 
instruction) 


Probe: If respondents mention differentiated instruction, probe for difference 
between true differentiation and small group instruction, i.e., providing different 
content and/or instructional techniques specifically tailored to meet students’ 
individual educational needs and/or learning styles 
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o Core Comprehension Strategies(Summarizing, Visualizing, Questioning, 
Metacognition, Inferring, Predicting, Text structure) 


Oo Core Comprehension Techniques (for example:Marzano’s vocabulary, Words 
their Way, PRC2, KWL, Word Study/word sorting, Read Aloud/Think Aloud, 
Reading Response, INSERT notes, PLAN, ReQuest, List-Group-Label, etc.) 


3. We would like to know more about the process that you, teachers and students use to select 
appropriate student reading materials. 


a. Do you use fluency snapshots and interest inventories to guide students’ selection of reading 
materials for their wide reading? Please provide examples. 


b. Are student reading levels used to help select appropriate materials? Please explain and 
provide examples. 


c. To what extent do students know how to select materials for wide reading that are appropriate 
to their abilities? Please explain and provide examples. 


Next, I would like to ask you a few questions about the specifics of when you meet with other 
teachers to plan and coordinate instruction and student groupings for the purpose of providing 


differentiated instruction that is appropriate to each student’s needs. 


4. Does your school have Grade-Level Teams(“horizontal” teams consisting of staff across subject 


areas from the same grade)? 


“Yes LINo (IF NO, SKIP TOQS) 
a. Are you all involved in these teams? 
b. What is your role in these teams? What topics are discussed in these meetings? 


5. Does your school have a Literacy Team (a “vertical” team focusing on literacy issues across 
grade levels)? 


“Yes (No (IF NO, SKIP TO Q6) 
f. Are any of you involved in this team? (Yes (No (IF NO, SKIP TO Q6) 
g. What is your role in this team? What topics are discussed in these meetings? 


6. We would like toknow more about your collaboration with the Literacy Intervention Teacher 
and how you plan together to provide blended, differentiated instruction. 


a. In which setting(s) do you meet or collaborate with the LIT? 
O One-on-one meetings: 
Oo Grade-level team meetings: 
oO Literacy team meetings: 


b. Do you collaborate with the LIT in scaffolding instruction (comprehension, vocabulary and 
fluency)? Please provide examples. 
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c. Do you plan together with the LIT for blended instruction that differentiates small group 
work? Please provide examples. 
0 Do you collaborate with the LIT to use assessment or other data to fine-tune 
differentiated instruction? Please provide examples (Probe for examples of how 
instruction is differentiated). 


d. Please describe other topics you discuss when meeting with the LIT to ensure that Tier 2 and 
3 students receive appropriately targeted, differentiated instruction during classroom push-in 
sessions and AMP after-school lessons. 


e. Have there been any changes in the strategies, activities and resources that you use during 
these push-in sessions, as compared to last year? 
oO If yes, please describe. 
oO Why did these changes come about? 


f. What challenges, if any, have you encountered while working with the LIT and/or Tier 2 and 
3 students this year? 
oO How were or will these challenges be addressed? 


7. We would like to learn more about your collaboration with the District Coordinator. 
a. Do you meet with him/her? If so, how often do you meet with him/her? 
___ Oncea ___ Several ___ Once ___ Several ___ Daily or 


month or less times a month a week times a week almost daily 


b. In what ways do you collaborate? What topics do you discuss? In what ways does he/she 
support your implementation of Striving Readers? 


c. Is there any additional support they could provide you with? 
8. In what other ways, on your own or in collaboration with the LIT, do you use student data? 
(This might include assessment data as well as other types of student data such as demographic 
and behavioral records.) 


a. Which kinds of data are you using? 


b. In which setting(s) (e.g., individually, as part of grade-level teams, as part of the Literacy 
Team, other) 


c. For what purposes? Please describe and provide examples. 


Probe & request examples: 

oO Plan for small group activities, 

O Monitor students' success in learning techniques for developing comprehension and using 
reading strategies appropriately, 

O Select appropriate materials at students' independent and instructional levels. 


We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 
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9. One of the components of Striving Readers is building your classroom libraries. 
a. How do you use the classroom libraries? 


Probe & request examples: 


For independent reading? 


For small group instruction? 


To support content area instruction? 


For read alouds? 


b. Do you use interest inventories to help students self select reading material and to guide your 
purchases? Please describe the process. 


c. How do you organize books in your classroom library? (Probe: Is there more than one 
criterion used to organize the libraries?) 


d. Have you or your students encountered any challenges in using your classroom libraries to 
support instruction? If so, please describe. 
Oo How have or will these challenges be addressed? 


10. Does your school have Listening Centers (where students can access models of fluency and 
record themselves to assess their own fluency)? [Yes (No (IF NO SKIP TO Q11) 


a. Where are these Listening Centers located? 

Li In the classrooms only 

(| Outside of the classrooms only (e.g., computer lab) 
Oo How accessible are they? 

1 Both 


b. Do you use the Listening Centers? [Yes ([INo 
c. (Ifnot used) Why not? 


d. (If used) How are you using them? 
O For which type of activities? 
fe) Are you using the Listening Centers with all students or subgroups of these 
students? 
fe) Please provide an example of how you use the Listening Centers to help 
differentiate instruction. 


Probe for evidence of true differentiation per above definition. 
e. (If used) Have you or your students encountered any challenges when using this technology? 


If so, please describe. 
Oo How were or will these challenges be addressed? 
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11. Do you have Media Centers (3 computers and 1 printer) in your classrooms)? 


“Yes [No (IF NO SKIP TO Q12) 


a. 


b. 


Do you use the Media Centers? Yes LINo 
(if not used) Why not? 


(If used) How are you using them? 
fe) For which type of activities? 
O Are you using the Media Centers with all students or subgroups of these students? 
O Please provide an example of how you use the Media Centers to help differentiate 
instruction. 


Probe for evidence of true differentiation per above definition. 
(If used) Have you or your students encountered any challenges when using this technology? 


If so, please describe. 
Oo How were or will these challenges be addressed? 


12. Do you and/or students in your school have access to Palm Pilots/Handheld Computers? 


(Yes (| No? If no, why not? (THEN SKIP TO Q13) 


Do you and/or your students use them in your classroom? [Yes LINo 
(if not used either by Teachers or by Students) Why not? 


(If used), how are they being used? 
oO For which type of activities do you use the Handheld Computers? 
o Are the Handheld Computers being used with all students or subgroups of 
students? 
oO Please provide an example of how you use the Handheld Computers to help 
differentiate instruction. 


Probe for evidence of true differentiation per above definition 
(If used) Have you or your students encountered any challenges when using this technology? 


If so, please describe. 
Oo How were or will these challenges be addressed? 


13. Overall, what are the strengths of the Striving Readers? 


a. 


In your opinion, what factors are helping to facilitate the implementation of Striving Readers 
as defined by the model during the current school year? At the school level? At the classroom 
level? 


A-180 


14. Overall, what challenges to implementing the Striving Readers initiative have you encountered 
during the current school year? 


c. How might these challenges be addressed? 
15. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2010 School Visits [New Schools] 
Non-ELA Content Area Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. Your school was among six schools that were selected for an 
in-depth case study in part because they were identified by the district as being successful in their 
implementation of the Striving Readers initiative. We are interested in learning about the implementation 
of this program in your school, overall and for each program component, identifying best practices, and 
gaining a better understanding of the facilitating conditions and challenges to implementation. Since we 
are requesting a lot of information and I know you have a busy schedule, please be as succinct as you can 
in your answers. You will have an opportunity to elaborate further at the end of the focus group. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 

recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 

specific comments that you would like to keep confidential, let us know and we will make sure they 

cannot be tied to yourself or your school. Is this all right? 

1. [Interviewer: please go around the room and ask participants to introduce themselves] Please 
introduce yourselves and tell us how long you have been a teacher in this school, and which 
subject area(s) and what grade level(s) you taught this year. 


We would like to know more about the types of support that you have received this year to help you 
integrate literacy into other content area instruction. 


2. Have you met with any of the following literacy experts? How often? What did you discuss with 
them? 


a. Literacy Intervention Teacher? 
b. Literacy Coach? 

c. Striving Readers Coordinator? 

d. Regular Language Arts teachers? 


e. Other staff member(s)? Please specify: 


A-182 


3. Have youever participated in any professional development conducted by the Striving Readers 
program related to integrating literacy into other content area instruction? LYes LINo 


e. If yes, when/what year did you start participating in this type of trainings? 


f. Which trainings did you participate in? (Probe and request specific examples: site-based 
professional development, Summer Institute and follow-up Institutes, Saturday Seminars) 


g. Which trainings were most useful and why? Which trainings were least useful and why? 
h. What areas or topics would you like to receive additional support or training in? 


4. What other kinds of support are available to help non-literacy staff integrate literacy into your 
content area instruction? 


5. What other kinds of support would you need to help you integrate literacy into your content 
area instruction? 


The next few questions are about the extent and relative success of your efforts tointegrate literacy 
as part of your content area instruction. 


6. When, if at all,did you start integrating literacy into your content area instruction? (Probe for: 
this year, last year, before Striving Readers began?) 


7. Have you been able to use any of the specific frameworks, techniques and strategies identified 
in the Striving Readers grant? 


c. Ifnot, how have you approached integrating literacy in your classroom? 


d. Ifyou have used specific Striving Readers frameworks, techniques and strategies, please 
provide examples. 


Probe and request examples: 


oO Striving ReadersInstructional Frameworks(for example: whole part whole 
instruction, differentiating instruction, independent reading, small group 


instruction) 


If respondents mention differentiated instruction, probe for difference between 
true differentiation and small group instruction , i.e., providing different content 
and/or instructional techniques specifically tailored to meet students’ individual 
educational needs and/or learning styles 


o Core Comprehension Strategies(Summarizing, Visualizing, Questioning, 
Metacognition, Inferring, Predicting, Text structure) 


Oo Core Comprehension Techniques (for example: Marzano’s vocabulary, Words 


their Way, PRC2, KWL, Word Study/word sorting, Read Aloud/Think Aloud, 
Reading Response, INSERT notes, PLAN, ReQuest, List-Group-Label, etc.) 
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8. Does your school have text setsdeveloped and distributed by the Striving Readers program 
(i.e.,collections of short books centered around specific content area themes, written at a variety of 
reading levels so students can access the books independently)with accompanying teacher guides? 


L)Yes 
b. 


“INo (IF NO SKIP TO Q9) 
Are the Striving Readers’ text sets being used in the content area classrooms? 


oO Social Studies 5 Not Used—why? o Used—how? 0 Don’t Know 
Oo Science 0 Not Used—why? 0 Used—how? o Don’t Know 


One of the components of Striving Readers is building your classroom libraries. 


d. 


How do you use the classroom libraries? To what extent do they support your efforts to 
integrate literacy into other content area instruction? 


Probe & request examples: 


For content area? 


For independent reading? 


For small group instruction? 


For read alouds? 


Do you use interest inventories to help students self select reading material and to guide your 
purchases? 


How do you organize books in your classroom library? (Probe: Is there more than one 
criterion used to organize the libraries?) 


16. To what extent have you used technology to support your efforts to integrate literacy into other 
content area instruction? 


Please describe and provide examples. [Probe: media centers (classroom computers and 
printers), listening centers, hand held computers] 


Have there been any changes in the use of technology as compared to last year? 
Have you and/or the students encountered any challenges when using technology? If so, 


please describe. 
oO How were or will these challenges be addressed? 


We would also like to know more about the factors facilitating or hindering the implementation of 
Striving Readers as it relates to the integration of literacy instruction into the other content areas. 


17. In your opinion, what factors are helping to facilitate the implementation of Striving Readers as 
defined by the model? At the school level during the current school year? 


14. Overall, what challenges to integrating literacy into the content areas have you encountered 
during the current school year? 


b. 


How might these challenges be addressed? 


15. Is there anything else you would like to add? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2010 School Visits [New Schools] 
Literacy Intervention Teacher (LIT) Interview Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. Your school was among six schools that were selected for an 
in-depth case study in part because they were identified by the district as being successful in their 
implementation of the Striving Readers initiative. We are interested in learning about the implementation 
of this program in your school, overall and for each program component, identifying best practices, and 
gaining a better understanding of the facilitating conditions and challenges to implementation. Since we 
are requesting a lot of information and we have a limited period of time, please be as succinct as you can 
in your answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. The recordings will not be shared with anyone outside of Metis; however, your comments 
may be used in reporting findings from this case study. If there are any specific comments that you would 
like to keep confidential, let us know and we will make sure they cannot be tied to yourself or your 
school. Is this all right? 


1. How long have you been an LIT? If you were in this role last year, has your role as the Literacy 
Intervention Teacher (LIT) in the Striving Readers Initiative changed from last year? 
a. Ifso, how? 
2. We would like to learn more about your collaboration with the District Coordinator. 
a. How often do you meet with him/her? 
___ Oncea ___ Several ____ Once ____ Several ____ Daily or 


month or less times a month a week times a week almost daily 


b. What topics do you discuss? In what ways does he/she support your implementation of 
Striving Readers? 


c. Is there any additional support they could provide you with? 
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Next, I would like to ask you a few questions about the specifics of when you meet with classroom 
teachers to plan, prioritize and coordinate instruction, responsibilities, and student groupings. 


3. Do you have one-on-one meetings with the teachers outside of instruction time? 


HYes “No (IF NO SKIP TO Q4) 
a. When do you have these meetings? 
___ Once a ____ Several ____ Once ___ Several ___ Daily or 
month or less times a month a week times a week almost daily 


b. Please describe how these meetings are used to ensure that Tier 2 and 3 students receive 
appropriately differentiated instruction during the classroom targeted intervention and/or after 
school classes.(Interviewer: be sure respondent understands “differentiation” means, more 
than small group structures, refers to providing different content and/or instructional 
techniques specifically tailored to meet students’ individual educational needs and/or 
learning styles) 


c. What topics are discussed in these meetings? 


d. Do you discuss assessment data other types of student data (such as demographic data or 
behavioral records) in these meetings? If so, for what purposes 


4. Does your school have Grade-Level Teams (“horizontal” teams consisting of staff across subject 
areas from the same grade)? 


“Yes LINo (IF NO, SKIP TO Q5) 
a. Are you involved in these teams? [Yes (INo (IF NO, SKIP TO Q5) 


b. Please describe how these meetings are used to ensure that Tier 2 and 3 students receive 
appropriately targeted, differentiated instruction during the classroom targeted intervention 
and/or after school classes. [If applicable]: How does this function differ from that of the 
one-on-one meetings?(Interviewer: be sure respondent understands “differentiation” means, 
more than small group structures, refers to providing different content and/or instructional 
techniques specifically tailored to meet students’ individual educational needs and/or 
learning styles) 


c. Do these teams use assessment data, orother types of student data (such as demographic data 


or behavioral records)? If so, for what purposes? [If applicable]: How does this differ from 
the use of data during one-on-one meetings? 
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5. Does your school have a Literacy Team (a “vertical” team focusing on literacy issues across 
grade levels)? 


Yes (No (IF NO, SKIP TO Q6) 
Are you involved in this team? “Yes (CINo (IF NO, SKIP TO Q6) 


Please describe how these meetings are used to ensure that Tier 2 and 3 students receive 
appropriately targeted, differentiated instruction during the classroom targeted intervention 
and/or after school classes.[If applicable]: How does this function differ from those of the 
one-on-one and Grade-Level Team meetings?(Interviewer: be sure respondent understands 
“differentiation” means, more than small group structures, refers to providing different 
content and/or instructional techniques specifically tailored to meet students’ individual 
educational needs and/or learning styles) 


Do these teams use assessment data, orother types of student data (such as demographic data 
or behavioral records)? If so, for what purposes? [If applicable]: How does this differ from 
the use of data during one-on-one and Grade-Level Team meetings? 


6. Have you encountered any challenges in collaborating with classroom teachers one-on-one or as 
part of the team meetings? 


a. 


If so, how were or will these challenges be addressed? 


7. I would like to know more about the targeted intervention and the work that you do with Tier 2 
and Tier 3 students during the regular school day. 


h. 


Which grades do you work with? How many classes? For how long (e.g., length of targeted 
instruction, number of periods per week per class)? 


On average, how many kids are there in the targeted group per class? 


How do you determine the needs of your students that might impact on their literacy 
development? 


How do you develop appropriate instruction for them? 


What types of strategies, activities and resources do you use when you meet with your tier 2 
& 3 students? 
O What types of student grouping do you use? 
O What strategies do you use to differentiate instruction for students of different 
ability levels within this group? (Interviewer: be sure respondent understands 
“differentiation” per above definition) 


Have you encountered any challenges when working in the language arts classroom with Tier 
2 and 3 students and/or their teachers this year? If so, please describe. 
oO How were or will these challenges be addressed? 


Have there been any changes in the strategies, activities and resources that you use during 
these targeted intervention sessions, as compared to last year? 

0 Ifyes, please describe. 

oO Why did these changes come about? 


A-187 


8. I would like to know more about the intensive intervention and the work that you do with Tier 
3 students during the after school program. 


g. 
h. 


What is your role in the after school component of Striving Readers? 


Please describe the structure and content of the after school Achieving Maximum Potential 
(AMP) programming for struggling readers. 


Are you using the AMP intervention software during the afterschool program? (1Yes [No 
oO (Ifnot used) Why not? 


oO (Ifused) How is it being used? 
1. For which type of activities? 
2. Are you using it with all students or subgroups of students? 


oO (Ifused) Have you, students, and/or teachers encountered any challenges when 
using this technology? If so, please describe. 
" How were or will these challenges be addressed? 


oO (Ifused) In what ways, if any, has this technology improved instruction and student 
learning in language arts? 


What successes has the school had with the after school component of Striving Readers? 


What challenges has the school encountered with the after school component of Striving 
Readers? 
oO How were or will these challenges be addressed? 


Have there been any changes in the strategies, activities and resources that you use during the 
afterschool program, as compared to last year? 

oO If yes, please describe. 

Oo Why did these changes come about? 


We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 


9. Do your teachers utilize the professional library in the school? If so, how? 


10. In what ways are the classroom libraries being used by the teachers? For what types of 
activities? 


What is your role, if any, in helping teachers use their classroom libraries? 


A-188 


11. Does your school have Listening Centers (where students can access models of fluency and 
record themselves to assess their own fluency)? [Yes [No (IF NO SKIP TO Q12) 


e. 


Do you use the Listening Centers to support your role in providing differentiated instruction 
to struggling readers? (Interviewer: be sure respondent understands “differentiation” per 
above definition) 


“Yes LINo 
(if not used) Why not? 


(if used) How are you using them? 
fe) For which type of activities? 
O Are you using the Listening Centers with all struggling students or subgroups of 
these students? 
fe) Please provide an example of how you use the Listening Centers to help 
differentiate instruction. 


Probe for evidence of true differentiation per above definition. 
(If used) Have you, students, and/or teachers encountered any challenges when using this 


technology? If so, please describe. 
Oo How were or will these challenges be addressed? 


12. Does your school have Media Centers (3 computers and 1 printer in the classroom)? || Yes 
LINo UF NO SKIP TO Q13) 


e. 


Do you use the Media Centers to support your role in providing differentiated instruction to 
struggling readers? (Interviewer: be sure respondent understands “differentiation” per above 
definition) 


LYes LINo 
(if not used) Why not? 


(If used) How are you using them? 
fe) For which type of activities? 
O Are you using the Media Centers with all struggling students or subgroups of these 
students? 
O Please provide an example of how you use the Media Centers to help differentiate 
instruction. 


Probe for evidence of true differentiation per above definition. 
Have you, students, and/or teachers encountered any challenges when using this technology? 


If so, please describe. 
oO How were or will these challenges be addressed? 
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13. Do you and/or students in your school have access to Palm Pilots/Handheld Computers? 
‘“lYes (No Ifno, why not? (THEN SKIP TO Q14) 


g. Do you use the Handheld Computers to support your role in providing differentiated 
instruction to struggling readers? (Interviewer: be sure respondent understands 
“differentiation” per above definition) 


LYes LINo 
h. Do your students use the Palm Pilots/Handheld Computers? “Yes [INo 
i. (if not used either by LIT or by Students) Why not? 


j. (fused) Are they being used during the afterschool program? 1 Yes [INo 
o If yes, how are they being used? 

1. For which type of activities do you use the Handheld Computers? 

2. Are the Handheld Computers being used with all struggling readers or 
subgroups of students? 

3. Please provide an example of how you use the Handheld Computers to help 
differentiate instruction. 

Probe for evidence of true differentiation per above definition 


k. (fused) Are they being used as part of the targeted intervention during the regular school 
day? “Yes No 


o If yes, how are they being used? 
"For which type of activities do you use the Handheld Computers? 
= Are the Handheld Computers being used with all struggling readers or 
subgroups of students? 
" Please provide an example of how you use the Handheld Computers to help 
differentiate instruction. 
Probe for evidence of true differentiation per above definition 


1. Have you, students, and/or teachers encountered any challenges when using this technology? 

If so, please describe. 
o How were or will these challenges be addressed? 
14. Overall, what are the strengths of your school’s literacy curriculum? 

a. In your opinion, what factors are helping to facilitate the implementation of Striving Readers 
as defined by the model? At the school level during the current school year? At the classroom 
level? 

15. Overall, what challenges to implementing the Striving Readers initiative have you encountered 
during the current school year? 

b. How might these challenges be addressed? 


16. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2010 School Visits [New Schools] 
Principal Interview Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. Your school was among six schools that were selected for an 
in-depth case study in part because they were identified by the district as being successful in their 
implementation of the Striving Readers initiative. We are interested in learning about the implementation 
of this program in your school, overall and for each program component, identifying best practices, and 
gaining a better understanding of the facilitating conditions and challenges to implementation. Since we 
are requesting a lot of information and I know you have a busy schedule, please be as succinct as you can 
in your answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. The recordings will not be shared with anyone outside of Metis; however, your comments 
may be used in reporting findings from this case study. If there are any specific comments that you would 
like to keep confidential, let us know and we will make sure they cannot be tied to you or your school. Is 
this all right? 


1. How long have you been the principal in this building? [If more than one year] Has your role 
in the Striving Readers Initiative changed from last year? 
a. Ifso, how? 


2. Does your school have a Literacy Team?(Note for the interviewer: ensure that literacy team is 
understood as a “vertical” team focusing on literacy issues across grade levels) 
“Yes “No (IF NO, SKIP TO Q3) 


a. Which of your staff are members of the Literacy Team? 


1] Principal 1] Grade level teacher(s) (| Literacy Intervention Teacher 
(| Librarian(s) [|] ELL/ESL Teacher(s) 
(| Special education teacher(s) (| Other: 


b. How often does the Literacy Team meet? 


_| Has not met (| Less than once per month (| Once per month 


(| Biweekly (| Weekly (| Several times a week or more 
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c. What role does the Literacy Team play at your school? 
d. What role do you play in the Literacy Team? 
e. How does the team address the needs of struggling readers? 
3. Does your school have Grade-Level Teams?(Note for the interviewer: ensure that grade level team 


is understood as a “horizontal” team consisting of staff across subject areas from the same grade.) 
(Yes (No (IF NO, SKIP TO Q4) 


a. Which of your staff are members of the Grade-Level Teams? 


b. How often do these teams meet? 


_| Have not met (| Less than once per month (| Once per month 


(| Biweekly (| Weekly (| Several times a week or more 


c. What role do these teams play at your school? 


d. How do these teams address the needs of struggling readers? 


We would like to learn more about the use of assessment data and how that impacts instruction. 


4. In what ways, if any, is your school using assessment data beyond mandated reporting to the 
district and state? 


a. Please describe and provide examples, including distinctions between how different teams or 
individuals use the data. 

oO Student placement in specific groups, programs, etc. 

0 Differentiating instruction (i.e., more than small group structures, this refers to 
providing different content and/or instructional techniques specifically tailored to 
meet students’ individual educational needs and/or learning styles.) 

oO Planning professional development 

o Other 


b. By whom? 
oO. Principal 
oO Literacy Team(s) 
O Grade-Level Teams 
o Other 
5. Are any other types of data (e.g., demographic, behavioral) being used? 
a. By whom? 


b. For what purposes? Please describe and provide examples. 


6. The AMP after school program is part of Striving Readers. We want to know more about your 
school’s use of this program. 
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f. Do you feel that the AMP program is appropriate to the reading levels of the students who are 
currently participating? Are there any students in AMP who you feel should not be there? 
(Reading levels too high/too low?) Are there students who are not in AMP who should be? 


g. Are the LIT or other after-school teachers using the AMP activities and materials? Are they 
using any supplemental materials or strategies? 


h. Are students using the AMP software? LYes LINo 
o Ifnot, why not? 


i. What successes has the school had with implementing the AMP program strategies, including 
the use of the AMP software? 


j. What challenges has the school encountered with the AMP program and software? 


We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 


7. To what extent has the use of technology been integrated into literacy instruction? 


a. Please describe and provide examples. [Probe: media centers (classroom computers and 
printers), listening centers, hand held computers] 


b. Have there been any changes in the use of technology as compared to last year? 


c. Have the students and/or teachers encountered any challenges when using technology? If so, 
please describe. 
Oo How were or will these challenges be addressed? 


d. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 


8. Now I would like to ask about your school’s efforts, if any, to integrate literacy into the content 
areas. 


a. Please describe your school’s efforts, if any, to integrate literacy into the content areas. 


b. Does your school have text sets developed and distributed by the Striving Readers program 
(i.e., collections of short books centered around specific content area themes, written at a 
variety of reading levels so students can access the books independently) with accompanying 


teacher guides? 
LlYes LINo 
oO Ifyes, are the Striving Readers text sets being used in the content area classrooms? 


Social Studies 0 Not Used—why? 5 Used—how? o Don’t Know 
Science 0 Not Used—why? 0 Used—how? o Don’t Know 
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c. Do non-literacy staff participate in professional development for the Striving Readers 
project? (Includes: Bilingual, SPED, Math, Science, Social Studies teachers) Yes LINo 


oO If yes, who has received professional development? Please specify staff positions. 
oO What topics were covered? 


d. What other kinds of support are there to help non-literacy staff integrate literacy into their 
content areas? 


e. What are some of the challenges that these teachers are facing? 
Oo How might these challenges be addressed? 


Now we would like to know more about different types of support that your school has received for 
the implementation of Striving Readers. 


9. 


10. 


11. 


12. 


13. 


14. 


15. 


Please describe your school’s partnership with National-LouisUniversity and Donna Ogle. 
a. In what ways has her support been helpful to the implementation of Striving Readers? 
b. Is there any additional support she could provide you with? 


What type of support has the District Coordinator provided to you and your school? Please 
describe. 


a. To what extent has his or her support been helpful for the implementation of Striving 
Readers? Please explain. 


b. Is there any additional support they could provide you with? 


Has your role as instructional leader changed as a result of your participation in Striving 
Readers? If so, how? 


Overall, what are the strengths of your school’s literacy curriculum? 


In your opinion, what factors are helping to facilitate the implementation of Striving Readers as 
defined by the model during the current school year? At the school level? At the classroom level? 


Overall, what challenges to implementing the Striving Readers initiative have you encountered 
during the current school year? 


a. How might these challenges be addressed? 
Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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Appendix A-2: Year 3 Measures 


This Appendix includes copies of the following surveys, instruments, and protocols used in the 
evaluation of the Chicago Striving Readers program: 


e Surveys 
Spring 2009 Literacy Improvement Survey for Teachers — Treatment Schools 
Spring 2009 Survey of Literacy Intervention and AMP Teachers 
Spring 2009 Literacy Improvement Survey for Librarians — Control Schools 
Spring 2009 Literacy Improvement Survey for Content Area Teachers — Treatment 
Schools 
Spring 2009 Literacy Improvement Survey for Teachers — Control Schools 
Spring 2009 Literacy Improvement Survey for Librarians — Control Schools 
Spring 2009 Literacy Improvement Survey for Content Area Teachers — Control Schools 
e Interview Protocols: 
Principal Interview Protocol — Treatment Schools 
Principal Interview Protocol — Control Schools 
Project Director Interview Protocol 
Literacy Advisor Interview Protocol 
School Coordinator Interview Protocol 
District Technology Coordinator Interview Protocol 
e Case Study 
Observation Protocol 
Self Contained Teachers Focus Group Protocols (Fall ’08, Spring ’09) 
ELA Teachers Focus Group Protocols (Fall ’08, Spring ’09) 
Non-ELA Teachers Focus Group Protocols (Fall ’08, Spring ’09) 
LIT Interview Protocols (Fall ’08, Spring ’09) 
Principal Interview Protocols (Fall ’08, Spring ’09) 
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Chicago Public Schools (CPS) Striving Readers 
Spring 2009 Literacy Improvement Survey for Teachers 


The following is a survey designed to gather your feedback on the essential components of the Striving Readers 
program. Survey results will be reported in the aggregate only. We will not use your name or identify individual 
respondents. Your feedback is extremely valuable to the success of this program. If you have questions about this 
survey, please contact Rebecca Swann at rswann@metisassoc.com or 212-425-8833. 


49, What is your primary role or teaching assignment? 


(Select the single best option. 
- If you are a Literacy Intervention Teacher (LIT), please select that role even if you also have other duties. 
- If you teach English language arts as well as other subject area(s), please respond to the survey with your 
role as an ELA teacher in mind.) 


Literacy Intervention Teacher (Link to LIT Survey) 
General education teacher (self-contained classroom teacher) (Continue with LIST Q3) 


English language arts teacher 
Teach English language arts and other academic subject — 
Teach other academic subjects but notEnglish language arts 
Bilingual/ELL teacher 
Special education teacher 
Librarian 
Reading specialist 
Other (please specify): 

Does this role include teaching of English language arts? 


e 6 Yes 
e No 


50. What other subjects do you teach? 
Oo Math 
Oo Science 
O Social Studies/Humanities 
O Other (please specify) 
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Comprehensive Instruction 


1. How often do you use the following practices to help students increase reading comprehension? 


than | times | 23 | 48 
Use of Instructional Practices Never times a | times a 
beech i eek week 
month | month | ™ 
Explicit instruction in use of summarizing as a 4 q 4 4 q 
comprehension strategy 
Explicit instruction in use of questioning as a comprehension 5 5 4 oq 4 
strategy 
Explicit instruction in use of predicting as a comprehension 5 q 4 4 5 
strategy 
Explicit instruction in using text structure (the organizational 
arrangements used to present information) as a 0 0 0 o o 
comprehension strategy 
Explicit instruction in use of visualization as a 5 4 5 q q 
comprehension strategy 
Explicit instruction in use of inferring as a comprehension q q 4 4 
strategy 
Explicit instruction in use of metacognition (students select 
appropriate comprehension strategies) as a comprehension 0 0 0 o 0 
strategy 
Establishing the purpose for reading. 0 0 0 o 0 
Monitoring students’ comprehension through questioning. o o o o o 
Making connections to background knowledge. 0 0 0 o 0 
Making connections between texts. 0 0 0 o o 
Synthesizing information within text or across texts. 0 0 0 im) 0 
Using differentiated instruction (using different instructional 
methods, modalities, materials, etc. for different groups or Oo o o Oo o 
individuals to address unique learning needs) 
Use of before, during, and after (BDA) reading strategies for 
comprehension instruction (A student constructed mental 
framework for reading begun before reading even begins, 0 0 40 o 0 
strengthened as students interact with the text during the 
reading, and reflected upon after reading.) 
Use of the gradual release of responsibility model for reading 
comprehension instruction (Leading students from ‘Modeled q q q qo q 
instruction’ to ‘Shared instruction’ to ‘Guided practice’ and 
finally ‘Students’ independent practice’) 
Using Pairing for Partner Reading & Content Too (PRC2) for q 5 5 qo q 


comprehension instruction 
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2. How often do you use the following practices to help students build their vocabulary knowledge? 


Less 1-3 
than times a ay 
Use of Instructional Practices Never times a | times a 
ce ‘i eek week 
month | month | ™ 
Explicit instruction in vocabulary 0 0 0 0 0 
Modeling the use of word parts 0 0 0 o 0 
Review of vocabulary words 0 0 0 o 0 
Use of vocabulary notebooks 0 0 0 o o 
Use of the PRC2 for vocabulary development. 0 0 0 o 0 
Use of before, during, and after (BDA) reading strategies for q 5 q qo q 
vocabulary instruction 
Use of the gradual release of responsibility model for q q q o q 
vocabulary instruction 
3. _ How often do you use the following practices to help students develop fluency? 
tian | times | 13 | 45 
Use of Instructional Practices Never times a | times a 
Heise é eek week 
month | month | ™ 
Teacher read aloud 0 0 0 0 0 
Teacher interactive read aloud 0 0 0 0 0 
Shared reading (students and teacher take turns in reading) og o Oo Oo oO 
Modeling reading for students 0 0 0 o o 
Explicit instruction in guided oral reading i) 0 i) o 0 
Focusing instruction on proper and meaningful phrasing i) 0 0 o 0 
Use of the PRC2 for fluency instruction. 0 0 0 o o 
Students listen to audio books, play aways 0 0 0 o o 
Use of the gradual release of responsibility model for fluency 5 5 q qo q 


instruction 
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4. How often do you use the following techniques to help students develop better reading strategies and skills? 


Techni Never/ Not Less than 1-3 times | 1-3 times | 4-5 times a 
ai Leds Familiar once a month | a month a week week 
Everybody Reads To (ERT) 0 0 0 0 o 
Exclusion Brainstorming 0 0 i) 0 o 
List-Group-Label 0 0 0 o o 
Predict-Locate-Add-Note (PLAN) 0 0 0 0 o 
ReQuest o o o o Oo 
Interactive Notation System for Effective Reading 
and Thinking (INSERT) 7 P a a - 
ABC Graffiti 0 0 0 0 o 
Guided Reading and Summarizing Procedure 
(GRASP) d A 0 2 ie 
5. _How often do you use the following grouping structures in your classes? 
: te an 1-3 times a 1-3 times a 4-5 times Multiple 
Grouping Structures Never once a times a 
month week a week 
month day 

Whole class/Large group oO Oo o Oo o oO 

Individual Work o o o oO o o 

Small groups or Pairs o o oO o oO oO 


6. Considering your own instruction (not that of the LIT or other instructors in your classroom), how often do you 
apply differentiated instruction (using different instructional methods, modalities, materials, etc. for different 
groups or for different individuals within groups, to address the unique learning needs of different students) in 


your classroom? 


Almost 
‘ About half Most of every 
Neves Rarely Siecastonally the time the time lesson or 
activity 
0 0 Oo Oo o 0 
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Purposeful Assessment 


7. Indicate how you use the data from the following assessments. (Please check all that apply.) 


Progress Assess 


Assessments Not Using Screening Diagnostic | Benchmarking Monitoring Outcomes 


Reading Benchmark 4 oO o o o 0 
Assessment 


Illinois Standards Achievement 
Test 


Basic Reading Inventory (BRI) 


Informal assessments 


Fluency Snapshots 


Spelling Inventories 


Other: 


Other: 


OQ;O;QO;Q;O7;,0/;0; a 


QOQ)QO;/O/;/0O;/0;0);0; a 
QOQ)O);QO,;0/;0;0);0; QO 
QOQ)O;/O/;/O;/0;0);0; a 
Q;)/Q;);Q0,;Q0/;0;0);Q0; QO 


OQ;ag;QO;Q;O7;0/;0); a 


Other: 


Data-Driven Instruction 


8. Please indicate the extent to which you use student assessment data for each of the following purposes. 


Use of Data Not at All To Some extent oa: fe ian To a Large Extent 


Placing students in intervention 
programs. 


Differentiating instruction. 


Identifying skills that need to be taught 
or retaught. 


Monitoring student reading progress. 


QO;O; O |;QO; Oa 
QO);O; O |;QO; Oa 


O);O); O |Q 
O;/QO; O |/Q 


Creating instructional groups (in-class). 


Grade-Level Teams 
9. Do you currently have grade-level (horizontal) teams at your school? 


Oo Yes 
Oo No (lIfno, skip to Question 15) 
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10. Overall, rate the grade-level team’s ability to use classroom assessment data in the following ways. 


Use of Data 


Poor 


Fair 


Good 


Excellent 


Not Sure 


Address the literacy needs of all students. 


Oo 


o 


im) 


mi) 


o 


Address the needs of struggling readers. 


Formalize lesson plans. 


Identify students who are eligible for targeted 
interventions. 


Identify strengths. 


Identify teaching and learning strategies. 


Improve classroom practice. 


O;O;O) Q ;O;O 


OQ/;/O)O,;, Q ;QO/;0 


OQ;O;Q; OQ ;oO;a 


Q)O);O);) QQ ;O/;0 


O;QO/;O) Q ;O/;O 


Literacy Teams 


11. Do you currently have a (vertical) literacy team in place at your school? 


Oo Yes 
0 No (Ifno, skip to Question 17) 


12. Overall, rate the quality of the literacy team’s performance in the following areas. 


Performance Areas 


Poor 


Fair 


Good 


Excellent 


Not Sure 


Using assessment data to pinpoint the staff's 
professional development needs. 


0 


o 


im) 


0 


o 


Addressing the needs of all students. 


Addressing the needs of struggling readers. 


Addressing the needs of grade-level teams. 


Improving literacy instruction at your school. 


O;O;O);0 


O};O)O);0 


O;QO;oO);0 


QO;O;O);0 


O;O);O);0 
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School-wide Intervention Materials 


13. For each of the materials listed below, 
= Indicate how frequently you currently use the materials to teach literacy. 
= For those that you are using, rate how comfortable you are withusing these materials to support student learning in language arts. 


b) IF USED: 


a) Frequenc 
adi me Rate your comfort level 


i N/A 
Materials ; = Not Less than} 1to3 1 to3 4to5 1 4 3 4 5 
. i Han Currently) oncea times a times a times a Not at all Very 
plies Using month month week week Comfortable Comfortable 
Working) 


Listening center (Classroom CD 


orcassette player, read-along audio 0 o 0 0 0 o 0 o o o o 
books, playaways and headphones) 

Media center (classroom-based 

station with three computers and a 0 im) 0 i) i) im) i) 0 0 0 0 
printer) 

Text sets (content related books of 

different reading levels, genres and i) i) 0 i) i) 0 0 0 0 0 0 
subject themes) 

Classroom library 0 0 0 0 0 0 i) 0 0 0 0 
Vocabulary notebooks 0 o 0 0 0 o 0 o o o o 
Textbooks 0 im) i) i) 0 im) i) 0 0 0 0 
Reading response notebooks 0 o 0 0 0 0 0 Oo im) 0 Oo 
School library 0 im) 0 Oo Oo 0 0 0 0 0 0 
Other informational texts (other q 4 7 q 4 qo q q q q q 


than text sets) 
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14. For each of the materials listed across the top of the chart below, please indicate which literacy instructional goals are supported by your use of that material 


in your classroom. (check all that apply.) 


Instructional goals that each material 
is used to support: 

Not Using 

Vocabulary Development 

Fluency 

Reading Comprehension 

Writing Skills 

Word Parts 

Word Recognition 

Spelling 

Grammar 

To teach content themes 

To develop students’ self-directed 

learning 

To supplement students’ textbook 

reading 

Teaching students to identify and use 

text structure 

Teaching students to identify and use 

the organizational features of 

expository writing 


Listening 
centers 


Media 
centers 


t 


Materials 


Classroom 
Text sets library 
J tL 


a) 


Q 


Q 


Vocabulary 
notebooks 
J 


Textbooks 


° 


To activate students’ priorknowledge | OO | OF | OG | GO | GO FT Oo 
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15. For each of the materials listed below, please indicate which literacy instructional practices are 
supported by your use of that material in your classroom. (Please check all that apply.) 


Instructional Practices 


: ae Differentiating 
: Not Differentiating ARNE ha Ae 
Materials Usi Guided Partner Individual Book club instruction for : 
sing : : : ; ‘ : English language 
reading reading reading discussions | struggling readers : 
(Tiers 2 & 3) learners/special 
education students 
Listening centers i) i) 0 i) 0 0 0 
Media centers 0 0 0 0 0 0 0 
Text sets 0 0 0 0 0 0 0 
Classroom library 0 0 0 0 0 0 0 
Vocabulary notebooks 0 0 0 0 0 o o 
Textbooks 0 0 0 0 0 0 0 
Reading response 4 q 5 5 q q q 
notebooks 
16. For each of the materials listed below, please indicate which grouping strategies are supported by your 
use of that material in your classroom. (Check all that apply) 
ae Grouping Strategies 
' 0 
Materials Using Whole Class/ Small Group/ Individual 
Large Group Pairs Work 
Listening centers i) 0 0 0 
Media centers 0 0 0 o 
Text sets 0 0 0 im) 
Classroom library im) im) 0 o 
Vocabulary notebooks 0 0 o o 
Textbooks 0 0 0 im) 
Reading response notebooks 0 0 0 o 
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Use of Handheld Computers 


17. a) Do you use handheld computers (Palm Pilots) to teach literacy? 
Yes (Skip to Question20.) 
No 


b) If you are NOT yet using handheld computers, please indicate why you are not using them below 
(Check all that apply) and then SKIP to Question 28: 


I have not received the handheld computers. 


Some or all of the computers are not working properly. 


Some or all of the necessary software applications have not been installed on the computers. 


I have not received sufficient professional development to feel comfortable using them. 


Because the Striving Readers program provides only 10 computers per classroom, and I do not 
like to have some students use them while others can not. 


I do not feel that they offer sufficient added benefit compared to traditional media (e.g. print, 
paper and pencil) to be worth the trouble. 


Other (please specify): 


18. Which grouping strategies do you support through the use of handheld computers (Palm Pilots) during 
literacy lessons? (Check all that apply) 
O Whole class/ Large group 
O Small group/pairs 
O Individual work 


19. How often do students use handheld computers (Palm Pilots) during literacy instruction in your 
classroom? 
o Less than once a month 
o 1-3 times a month 
o 1-3 times a week 
o 4-5 times a week 


20. Rate how comfortable you are withusing the Palm Pilotsto support your literacy instruction 


7 2 ; 4 5 
Not at all Very 
Comfortable Comfortable 
oO Oo oO Oo Oo 
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21. 


22. 


Which specific academic foci or instructional objectives do you support with the use of handheld 
computers (Palm Pilots)? (Check all that apply) 


Fluency 


Vocabulary development 


Developing students’ reading comprehension strategies 
Writing skills 
Word parts 


Word recognition 


Spelling 


Grammar 


Locating information 


Evaluating information 


Synthesizing information 


Organizing information 


To develop students’ self-directed learning 


Teaching students to identify and use the organizational features of expository writing 


To activate students’ prior knowledge 


Which of the following instructional activities and practices do you support with the use of handheld 
computers (Palm Pilots)? (Check all that apply) 


Monitoring distribution and completion of assignments 


Assessing students’ literacy skills 


Monitoring students’ progress 


Differentiating instruction for struggling readers (Tiers 2 & 3) 


Differentiating instruction for English language learners/special education students 


Guided reading 


Partner reading 


Individual reading 


Book club discussions 
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23. In the table below, please indicate your frequency and comfort of use of each software application when using the handheld computers for literacy 


instruction. 


a) 


Indicate how frequently students in your class currently use each software application on the Palm Pilots during literacy instruction. 
b) For those that you are using, rate how comfortable you are withusing each software application on the Palm Pilotsto support your 
literacy instruction. 


Handheld Computer 
Software Applications 


a) Frequency 


b) IF USING: 
Rate your comfort level 


Not 
Currently 
Using 


Less than 
once a 
month 


1to3 
times a 
month 


1to3 
times a 
week 


4to5 
times a 
week 


1 


Not at all 
Comfortable 


3 


5 


Very 
Comfortable 


CPS ON-Demand (Safari 
Montage). 


mi) 


Q 


QO 


Q 


Q 


Q 


Q 


iKWL 


Freewrite 


PiCo Maps 


Internet Reciprocal Teaching 


Flingit 


Viewpoint 


Sketchy 


MS Word 


MS Excel 


Slideshow to Go 


Cells 


PAAM management software 
application 


Go Manage 


QO; O jO;,O;,O;O;/QO);0/;0/;0);0;0/;0 


QO; O jOJO;QO;O;,O;/Q;O)/0/;0;0)/0 


QO; O jOJO;Q;O;,QO;/Q;O)/0O;0;0)/0 


QO; OU jOJO;Q;OV,QO;/Q;O);0/;0;0)/0 


QO; O jO;,O;,O;O;/QO);0;/0;/0);0;0/;0 


QO; O jOJO;O;OVQO;/Q;O);Q;0;0)/0 


QO; O jOVJO;O;O;QO;/Q;O);0/;0;0)/0 


QO; OU jOJO;Q;OVQO;/Qy;O),Q;Q0;0);0; a 


QO; OU jOJO;Q;OVQO;/Q;O;Q;/Q;0);0; a 


QO; OU jOJO;/Q;O;,QO;/Q;O)Q0;0;0);0; 2 


QO; O jOJO;Q;OVQO;/Q;O)/Q0/;0O;0)/0 
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24. For each of the handheld computer software applications listed below, please indicate which literacy instructional objectives are supported by your use of 
that application in your classroom. (Please check all that apply.) 


Handheld Computer 
Software 


Not Using 


Instructional Objectives supported by this application 


Fluency 


Vocabulary 
Development 


Reading 
Comprehension 


Writing 
Skills 


Word 
Parts 


Word 
Recognition 


Spelling 


Grammar 


CPS ON-Demand (Safari 
Montage). 


im) 


oO 


o 


O 


Oo 


iIKWL 


Freewrite 


PiCo Maps 


Intemet Reciprocal Teaching 


Flingit 


Viewpoint 


Sketchy 


MS Word 


MS Excel 


Slideshow to Go 


Cells 


OQ)O;QO;,O/;0)/0/;/0/0/0/;/0/;0; a 


Q)Q;/Q/Q0/0/0/0/0/0/0/0/; a 


O;O);Q;O;/0O)0O/0/;/0)0/0/0 


O)/Q;QO/;0/0/;/0/0)/0;0/0)0 


OQ);O,Q;O;0O);O0/0/;0);0/0/0 


OQ)/Q;QO/;0/0/;0/0);/0;0/0)0 


OQ)Q;QO/;0;/0/;/0/0)/0;0/0)0 


OQ)/O,Q;O;/O)/O0/O0;/0);0/0;0) a 


OQ)OV,Q;O;/O)/O0/O0/;/0);0O/0;0) a 
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Classroom Library 


25. Please check the ways that you use your classroom libraries. (Check all that apply) 


For content area instruction 


[1 For independent reading 


For small group instruction 


For read alouds 


26. Do you use interest inventories to help students self select reading material? 
Oo Yes 
o No 


27. Do you use interest inventories to guide your purchases for the classroom library? 
o Yes 
o No 


28. Please indicate how true each of the following statements areabout the organization of books in your classroom 


library. 

Not At Slightly Somewhat Very 
My classroom library... All True True True True 
...1S easily accessible to students. 0 0 o o 
...1S well organized and in good shape. 0 0 0 o 
...has a checkout system in place. 0 0 0 0 
...includes a variety of reading materials that are q q q q 
appropriate for readers of differing abilities. 
.. includes a variety of texts that appeal to readers with q oq q q 
differing interests. 
...has reading materials grouped by genre. oO oO o o 
...has reading materials clearly labeled. o oO ) Oo 
..-has both nonfiction and fiction books. o Oo Oo Oo 


School Library 


29. How often do you take your class to the library? 
Oo Never 

Rarely (less than once a month) 

Sometimes (at least once a month) 

Often (at least once a week) 

Almost daily or daily 


30. To what extent do the library resources support the Striving Readers program? 


oO Notatall 

O Toasmall extent 

O Toamoderate extent 
o Toa large extent 

Oo Don’t know 


31. How does the librarian work with you? (Check all that apply.) 
Ol The librarian does not work with me. 


Ol The librarian provides resources for class projects. 


Ol The librarian and I collaborate on how to supplement lessons with library resources. 


A-209 


Ol Other (please specify): 


32. To what extent does the librarian consult withclassroom teachers in using Striving Readers library 


funds to order reading materials that are grade level and content appropriate? 


Oo Notatall 

O Toasmall extent 

O Toamoderate extent 
o Toa large extent 

O Don’t know 


33. To what extent does the librarianconsider students’ needs and reading abilitieswhen ordering books 


and other reading material with Striving Readers library funds? 
o Notatall 

To a small extent 

To a moderate extent 

To a large extent 

Don’t know 


oo0o°o 


34. To what extent does the librarianconsider students’ interests and motivationwhen ordering books and 


other reading material with Striving Readers library funds? 
o Notatall 

To a small extent 

To a moderate extent 

To a large extent 

Don’t know 


35. How does the librarian work with your students? (Check all that apply.) 
Does not work with my students. 

Works with students on research skills. 

Directs students to resources tied to curriculum. 

Conducts read-alouds. 


(o) 


competitions, events). 
Assists students with class projects. 
Teaches students how to navigate Internet resources. 
Guides struggling readers to summer programs. 
Other (please specify): 


Collaboration with LIT 


36. How often do you meet or collaborate with the LIT in the following settings? 


Provides students with information about extracurricular academic activities (e.g., spelling bee, writing 


G ‘aia @uaiet N Less than 1-3 times 1-3 times a 4-5 times 
bas sasanamariaans ad once a month | amonth week a week 

Scheduled one-on-one meetings im) 0 0 0 im) 
Impromptu one-on-one meetings 
(during lunch, prep periods, o 0 0 0 o 
before/after school, etc.) 
Grade-level (horizontal) team q q 4 q q 
meetings 
Literacy (vertical and horizontal) qo q q oq q 


team meetings 
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37. To what extent has your collaboration with the LIT facilitated your efforts to use the following methods to 


support struggling readers in your class? 


Extent to which collaboration with LIT facilitated use of 


Instructional methods meds 
To a small To a moderate To a large 
Not at all 
extent extent extent 
a. Differentiating instruction im) 0 im) o 
b. Scaffolding of instruction im) 0 0 o 
c. Student groupings o 0 0 o 
d. Use of the Whole-Part-Whole instructional model o 0 0 o 
e. Using the media center o 0 o o 
f. Using listening centers o 0 0 o 
g. Using handheld computers o 0 0 o 
h. Using text sets o 0 0 o 
i. Using assessment data to monitor student progress 0 i) i) 0 
j. Using student assessment data for instructional q q q q 
planning 


38. To what extent has your collaboration with the LIT facilitated your ability to provide effective instruction in the 


following areasfor struggling readers? 


Extent to which collaboration with LIT facilitated effective 
Academic areas TasuEueuOn 
To asmall To a moderate To a large 
Not at all 
extent extent extent 
a. Comprehension 0 0 Oo Oo 
b. Fluency 0 0 Oo o 
c. Vocabulary 0 0 o o 
d. Writing skills im) 0 0 im) 
e. Word parts 0 0 Oo o 
f. Word recognition 0 0 o o 
g. Spelling 0 0 0 o 
h. Reading/literacy in content areas 0 0 0 o 


39. Overall, how effective has the literacy intervention teacher (LIT) push-in been in improving the reading skills of 


struggling readers in your classroom? 
Oo Notat all effective 

Minimally effective 

Somewhat effective 

Effective 

Very effective 


oo0o°o 
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Professional Development 


40. For each of the following Striving Readers professional development sessions conducted during the 2008-2009 


school year, please indicate: 


= Whether you participated, and 
= Ifso, how useful the session(s) was (were) in helping you support student learning in 


language arts 


os you If YES, how useful was the session? 
; : participate? 
Professional Development Sessions 
No Ves Not | Somewhat | Moderately | Extremely 
Useful Useful Useful Useful 

AMP Intensive Intervention Program q q 5 5 q qo 
Training 

2008 Summer institute im) 0 0 i) i) im) 
School-year follow-up institutes im) 0 i) 0 0 0 
Technology training (use of handhelds) im) 0 i) 0 0 0 
Training in LIT/ teacher collaboration o 0 0 0 0 o 
School-based professional development im) 0 i) 0 0 0 
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41. For each of the following topics, indicate: 


= Whether you received professional development addressing this topic during the current year 
= Ifso, rate the impact that professional development you received has had on your comfort with each teaching practice. 


If YES, what impact did the professional development have on your 


Reread ED? comfort with each teaching practice? 
Teaching practices 
No Yes No Impact Slight Impact Sea Major Impact 

Building academic vocabulary 0 im) im) i) o 0 
Classroom libraries 0 im) im) 0 0 0 
Creating literacy-rich classroom environments 0 0 0 i) im) 0 
Differentiating instruction 0 0 im) 0 im) 0 
Direct vocabulary instruction i) im) 0 i) o 0 
Incorporating text sets in your instruction 0 0 im) i) i) 0 
Increasing student motivation 0 im) im) 0 o 0 
Supporting students’ self-directed learning 0 o o 0 o o 
Using before, during, and after reading strategies 0 0 i) 0 o 0 
Using student assessments to guide instruction 0 o o 0 o o 
Using handheld computers (Palm Pilots) 0 im) im) i) 0 0 
Using literacy-based software 0 o 0 0 o o 
Using the PRC2 model 0 o o 0 o o 
Using the whole-part-whole classroom instruction model i) 0 im) i) 0 0 
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a 
PO 


apply) 

Marzano’s vocabulary 

KWL 

Word Study/word sorting 

Using PRC2 for fluency instruction. 

Using PRC2 for comprehension instruction. 

Using PRC2 for vocabulary development. 
Everybody Reads To (ERT) 

Exclusion Brainstorming 

List-Group-Label 

Predict-Locate-Add-Note (PLAN) 

ReQuest 

Interactive Notation System for Effective Reading and Thinking (INSERT) 
Read Aloud/Think Aloud 

ABC Graffiti 

Guided Reading and Summarizing Procedure (GRASP) 
Teachingsummarizing as a comprehension strategy 
Teachingquestioning as a comprehension strategy 
Teachingpredicting as a comprehension strategy 
Teachingtext structure as a comprehension strategy 
Teachingvisualization as a comprehension strategy 


Teachinginferring as a comprehension strategy 


OUOUOUOOOOOOOOOOOOOOOOOOLR 


Teachingmetacognitionas a comprehension strategy 


Struggling Readers: Extended Day (Afterschool) Intervention 


43. Do you teach the Striving Readers (AMP) after-school program? 
oO Yes (Link to AMP questions) 
o No 


About You [All types of respondents] 


44. What is the name of your school? 
[drop down list] 


. Please check the techniques in the list below for which you would like to receive more training. (Check all that 
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FULLER TALCOTT 
GALE COM TELPOCHCALLI 


GOMPERS VOLTA 
foray, 


45. At which grade level(s) are you teaching reading/English language arts this year (2008-09)? (Check 
all that apply): 


0K¥010203040506070809010011 012 


46. In which of the following settings do you teach literacy? (Check all that apply) 
0 Self-contained 

Subject-Area specialist 

Departmentalized 

Double block 

Other (Please specify): 


oo0o°o 


47, How many years have you been teaching? [INSERT TEXTBOX] 
48. How many years have you been teaching at this school? [INSERT TEXTBOX] 


49, How many years have you been teaching reading? [INSERT TEXTBOX] 


Teachers will be reimbursed by CPS-Striving Readers for their time to complete this survey. In order to be 
reimbursed we need you to identify yourself so that we can verify that you completed the survey. If you would like 
to be reimbursed, please provide your name and email address below, and be sure that you identified your school in 
the previous item. Your survey responses will still remain strictly confidential and will never be reported in any 
form that would allow anyone to connect your responses with your name. Providing this information is optional. 


Your Name: Email: 


Thank you for completing this survey! 
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Chicago Public Schools Striving Readers 
Spring 2009 Survey of Literacy Intervention and AMP Teachers 


The following is a survey designed to gather your feedback on the push-in intervention and AMP after-school components of 


the Striving Readers program for Tier 2 and 3 students. 


It will take you approximately 60 minutes to complete 


(approximately 30 minutes for AMP-only teachers). Results will be reported in the aggregate only; we will not use your 
name or identify individual respondents. Your feedback is extremely valuable to the success of this program. If you have 
questions about this survey, please contact Rebecca Swann at rswann@metisassoc.com or 212-425-8833. 


Please answer the following questions with regard to your work with students in the Targeted intervention 
group (i.e., additional instruction in small group setting for Tier 2-3 students). 


1. How often do you use the following grouping structures during the push-in intervention with Tier 2and 3 


students? 
: Ress ithan 1-3 times a 1-3 times a 4-5 times Muluple 
Grouping Structures Never once a times a 
month week a week 
month day 
Individual Work o o o o oO oO 
Small groups/Pairs o oO o o o o 


2. Duringyour work in the regular classroom with students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following practices to help struggling readers increase reading 


comprehension? 


Use of Instructional Practices 


Never 


Less than 
oncea 
month 


1-3 
times a 
month 


1-3 
times a 
week 


4-5 times 
a week 


Explicit instruction in use of summarizing as a comprehension strategy 


Q) 


im 


Oo 


o 


Q) 


Explicit instruction in use of questioning as a comprehension strategy 


Explicit instruction in use of predicting as a comprehension strategy 


Explicit instruction in using text structure (the organizational arrangements 
used to present information) as a comprehension strategy 


Explicit instruction in use of visualization as a comprehension strategy 


Explicit instruction in use of inferring as a comprehension strategy 


Explicit instruction in use of metacognition (students select appropriate 
comprehension strategies) as a comprehension strategy 


Establishing the purpose for reading 


Monitoring students’ comprehension through questioning 


Making connections to background knowledge 


Making connections between texts 


Synthesizing information within text or across texts 


O;/Q;O/;0)0; Q;QO/0) 0;0/0 


OQ)/Q;QO,;0);0; Q;QO/;/0); 0;0/;0 


OQ)/Q;O/;/0)0; 0 ;O/;/0) 0;0/0 


OQ,)Oy;O;O/;O; Q;O;O; Q;O/0 


Using differentiated instruction (using different instructional methods, 
modalities, materials, etc. for different groups or individuals to address unique 
learning needs) 


QO 


OQ j;O,O;O;0/0; 0 ;Q;0; Q);0/0 


Q 


Q 


Q 


Use of before, during, and after (BDA) reading strategies for comprehension 
instruction (A student constructed a mental framework for reading begun 
before reading even begins, strengthened as students interact with the text 
during the reading, and reflected upon after reading.) 


Use of the gradual release of responsibility model for reading comprehension 
instruction (Leading students from ‘Modeled instruction’ to ‘Shared 
instruction’ to ‘Guided practice’ and finally ‘Students’ independent practice’) 
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Less than 1-3 1-3 4-5 tim 
Use of Instructional Practices Never once a times a | timesa sear 
a week 
month month week 
Using Pairing for Partner Reading & Content Too (PRC2) for comprehension q 5 4 q 4 
instruction 
3. Duringyour work in the regular classroom with students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following practices to help struggling readers build their vocabulary 
knowledge? 
Less than 1-3 1-3 5 tines 
Use of Instructional Practices Never once a times a | timesa 
a week 
month month week 
Explicit instruction in vocabulary 0 0 0 0 0 
Modeling the use of word parts 
Review of vocabulary words 0 0 0 o 0 
Use of vocabulary notebooks 0 0 0 o 0 
Use of the PRC2 for vocabulary development 0 0 0 0 0 
Use of before, during, and after (BDA) reading strategies for vocabulary 5 4 4 q 4 
instruction 
Use of the gradual release of responsibility model for vocabulary instruction i) im) 0 0 0 
4. Duringyour work in the regular classroom with students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following practices to help struggling readers develop fluency? 
Less than 1-3 1-3 es ines 
Use of Instructional Practices Never once a times a | timesa 
a week 
month month week 
Teacher read aloud 0 0 0 0 0 
Teacher interactive read aloud 0 0 0 0 0 
Shared reading (students and teacher take turns in reading) o o Oo oO og 
Modeling reading for students 0 0 i) 0 0 
Explicit instruction in guided oral reading 0 0 i) i) i) 
Focusing instruction on proper and meaningful phrasing 0 0 im) 0 0 
Use of the PRC2 for fluency instruction 0 0 0 o o 
Students listen to audio books, play aways 0 0 0 o o 
Use of the gradual release of responsibility model for fluency instruction i) im) i) i) Oo 
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5. Duringyour work in the regular classroom with students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following techniques to help students develop better reading strategies and 


skills? 


Techniques 


Never/ Not Less than 1-3 times | 1-3 times 
Familiar once a month a month a week 


4-5 times a 
week 


Everybody Reads To (ERT) 


Oo o 0 


oO 


Exclusion Brainstorming 


List-Group-Label 


Predict-Locate-Add-Note (PLAN) 


ReQuest 


Interactive Notation System for Effective Reading 
and Thinking (INSERT) 


ABC Graffiti 


Guided Reading and Summarizing Procedure 
(GRASP) 


QOQ/;O); 0 ;o;o;o;0);0 


Q/;O); 0 ;o;o/;0o;0 


Q/;O; 0 ;o,;o/;0;0 


Q/;O) Q ;O;/O/0/0 


QOQ/;O; Oo ;o;o/;o0;0 


6. How often do you meet with ELA classroom teachers at the following grade levels to discuss instruction-related 
issues regarding your work with students in the Targeted intervention group. 


Frequency of Meetings with Grade-Level Teachers 


Less than 1-3 1-3 4-5 
Never once a times a | timesa | timesa 
month month week week 
Grade 6 teachers Oo Oo oO Oo Oo 
Grade 7 teachers Oo Oo Oo Oo o 
Grade 8 teachers o o o Oo o 


7. In which setting(s) do you meet or collaborate with ELA classroom teachers? (Check all that apply for each 


grade) 
Settings for Meetings with Classroom Teachers Grade 6 Grade 7 Grade 8 
a. Scheduled one-on-one meetings Oo ) 0 
b. Impromptu one-on-one meetings (during lunch, prep 4 q 5 
periods, before/after school, etc.) 
c. Grade-level team meetings 0 o o 
d. Literacy leadership team meetings im) oO =) 
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8. How often do you meet with SIXTH-GRADE classroom teachers to discuss implementing each of the 
following instructional methods for students in the in-class Targeted Intervention group (Tiers 2-3)? 


How often discussed with classroom teachers 
Instructional methods and activities for Targeted Less than AA ainkes ices ines. leds tance 
Intervention Never once a 
a month a week a week 
month 

Differentiated instruction 0 0 0 0 0 
Student groupings 0 0 0 o o 
Use of Striving Readers texts sets, text set teacher 

: : : 0 0 0 0 0 
guides, technology, classroom library, school library 
Use of specific Striving Readers comprehension q q q q q 
strategies for reading 
U sing specific Striving Readers instructional q 5 q q 4 
techniques for comprehension instruction 
U sing specific Striving Readers instructional 5 q 4 q 5 
techniques for vocabulary instruction 
Using specific Striving Readers instructional 
techniques for fluency instruction 
Discussing specific students’ reading progress 
Coordinating instruction between lessons for the 
whole class and lessons for the Targeted Intervention 0 0 0 0 0 
group 
Using student assessment data for instructional 4 4 q 4 4 
planning 


6. Please indicate the extent to which you use student assessment data for each of the following purposes related to 
your work with students in the Targeted intervention group? 


To some ee To a large 
Use of Data Not at All moderate 8 
extent extent 
extent 

Differentiating instruction 0 Oo 0 0 
Identifying skills that need to be retaught or 5 q 5 q 
retaught 

Monitoring student reading progress o 0 0 0 
Creating instructional groups (in-class) 0 i) i) 0 
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Schoolwide Intervention Materials 


7. Please use the table below to tell us about your use of various Striving Readers materials during your instruction of Tier 2 and 3 students in the regular 
classroom. Please rate the use of these materials only in reference to your own instructional activities; do not include those of the classroom teacher or other 
adults who may be assisting in the classrooms you serve. 


For each of the materials listed below, 
= Indicate how frequently you currently use the materials to teach literacy. 
= For those that you are using, rate how comfortable you are with using these materials to support student learning in language arts. 


ee b) IF USING: 
4 y Please rate your comfort level 
Materials N/A Not | Lessthan | 1to3 1to3 4to5 1 2 3 4 5 
(Do Not [Currently| oncea times a times a times a Not at all Very 
Have) Using month month week week Comfortable Comfortable 
Listening centers (Classroom 
CD & Cassette Player, Read- q q q q q q 5 5 5 q 5 
Along audio books, 
playaways, headphones) 
Media centers (three 4 5 nq 5 q q q q q q q 
computers and a printer) 
Text sets ( content related 
books of different reading q q q q q q q qo q q q 
levels, genres and subject 
themes) 
Classroom library 0 i) 0 i) i) i) 0 o 0 o 0 
Vocabulary notebooks 0 0 0 0 0 0 0 o o o o 
Textbooks i) 0 Oo 0 0 0 i) 0 i) 0 0 
Reading response notebooks 0 0 0 0 0 0 0 o o o o 
School library 0 0 0 0 i) 0 0 im) 0 0 0 
Other informational texts 
(other than text sets) 3 0 o rt = 7 a 0 = 
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Use of Handheld Computers During Targeted Intervention Instruction 


8. a) Do you use handheld computers (Palm Pilots) to teach literacyduring Targeted Intervention 
instruction of Tier 2 and 3 students in the regular classroom? 


Yes (Skip to Question 9.) 
No 


b) If you are NOT yet using handheld computers, please indicate why you are not using them below 
(Check all that apply) and then SKIP to Question 15: 


I have not received the handheld computers. 


Some or all of the computers are not working properly. 


Some or all of the necessary software applications have not been installed on the computers. 


I have not received sufficient professional development to feel comfortable using them. 


Because the Striving Readers program provides only 10 computers per classroom, and I do not 
like to have some students use them while others can not. 


They are being used primarily for whole class instruction. 


I do not feel that they offer sufficient added benefit compared to traditional media (e.g. print, 
paper and pencil) to be worth the trouble. 


Other (please specify): 


9. Which grouping strategies do you support through the use of handheld computers (Palm Pilots) during 
Targeted Intervention instruction of Tier 2 and 3 students in the regular classroom? (Check all that 
apply) 

O Whole class/ Large group 
O Small group/pairs 
O Individual work 


10. How often do your students use handheld computers (Palm Pilots) during Targeted Intervention 
instruction of Tier 2 and 3 students in the regular classroom? 
o Less than once a month 
o 1-3 times a month 
o 1-3 times a week 
o 4-5 times a week 


11. Rate how comfortable you are withusing the Palm Pilotsto support your literacy instruction during 
targeted intervention instruction. 


1 2 3 4 5 
Not at all Very 
Comfortable Comfortable 
Oo Oo oO Oo oO 
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. Which specific academic foci or instructional objectives do you support with the use of handheld 


computers (Palm Pilots) during Targeted Intervention instruction? (Check all that apply) 


Fluency 


Vocabulary development 


Developing students’ reading comprehension strategies 
Writing skills 
Word parts 


Word recognition 


Spelling 


Grammar 


Organizing information 


Locating information 


Evaluating information 


Synthesizing information 


Demonstrate knowledge of key concepts 


To develop students’ self-directed learning 


Teaching students to identify and use the organizational features of expository writing 


To activate students’ prior knowledge 


. Which instructional methods do you support with the use of handheld computers (Palm Pilots) during 


Targeted Intervention instruction? (Check all that apply) 


Monitoring distribution and completion of assignments 


Assessing students’ literacy skills 


Monitoring students’ progress 


Differentiating instruction 


Teaching comprehension strategies 


Using comprehension techniques 


Guided reading 


Partner reading 


Individual reading 


Book club discussions 
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14. In the table below, please indicate your frequency and comfort of use of each software application when using the handheld computers during your Targeted 


Intervention instruction of Tier 2 and 3 students in the regular classroom. 


a) Indicate how frequently your Tier 2-3 students currently use each software application on the Palm Pilots during literacy instruction. 
b) For those that you are using, rate how comfortable you are withusing each software application on the Palm Pilots to support your 
literacy instruction. 


Handheld Computer 
Software Applications 


a) Frequency 


b) IF USING: 
Rate your comfort level 


Not 
Currently 
Using 


Less than 
once a 
month 


1to3 
times a 
month 


1to3 
times a 
week 


4to5 
times a 
week 


1 


Not at all 
Comfortable 


3 


5 


Very 
Comfortable 


CPS ON-Demand (Safari 
Montage). 


mi) 


Q 


QO 


Q 


Q 


Q 


Q 


iKWL 


Freewrite 


PiCo Maps 


Internet Reciprocal Teaching 


Flingit 


ViewPoint 


Sketchy 


Ms Word 


Ms Excel 


Slideshow to Go 


Cells 


PAAM management software 
application 


Go Manage 


QO; O ;O;O;,O;O;/0);0/;0;/0);0;0/;0 


QO; O jOJO;QO;O;,O;/Q;O)/0/;0;0)/0 


QO; OU jOJO;Q;OV,QO;/Q;O)/Q0;0O;0)/0 


QO; OU jOJO;Q;OV,QO;/Q;O);0/;0;0)/0 


QO; O jOJQO;Q;O;,Q;/Q;QO;Q;/Q0;0)0 


QO; O jOJO;O;OVQO;/Q;O);Q;0;0)/0 


QO; O jOJO;Q;OVQO;/Q;O);Q/;O0;0)/0 


QO; OU jOJO;Q;O;,QO;/Q;O);Q;/0;0);0; a 


QO; OU jOJO;/Q;OVQO;/Q;O;Q;/Q;0O);0; a 


QO; OU jOJO;Q;O;,Q;/Q;O)QO;0;0);0; a2 


QO; O jOJO;Q;OVQO;/Q;O)/0/;0;0)/0 
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15. For each of the handheld computer software applications listed below, please indicate which literacy instructional objectives are supported by your use of 


that application during your Targeted Intervention instruction of Tier 2 and 3 students. (Please check all that apply.) 


Handheld Computer Software 


Not 
Using 


Instructional Objectives supported by this application 


Vocabulary 
Development 


Fluency 


Reading 
Comprehension 


Writing 
Skills 


Word 
parts 


Word 
recognition 


Spelling 


Grammar 


CPS ON-Demand (Safari 
Montage). 


Oo 


oO 


o 


oO 


im) 


iKWL 


Freewrite 


PiCo Maps 


Internet Reciprocal Teaching 


Flingit 


Viewpoint 


Sketchy 


Ms Word 


Ms Excel 


Slideshow to Go 


Cells 


QOQ)O,Q;0O;0) Q ;Q;Q;O);Q/;O0; 


Q)O;QO;O/0/; Q joy;oy;a;o;a 


Q)O),O;QO;0) OQ jO;O;O);0O/;0; a 


OQ)Q;O/;0O;/0; Q ;O;Q;Q/Q);0 


OQ)O,O/;O/;0) OQ jOos;O;O;0;0 


OQ)Q;O/;0O;0; Q ;oO;Q;Q/Q),0 


OQ)OQ;QO/;O0/;0/; QQ jo;o;ao;o;a 


Q)O,O;QO;0) OQ jO;QO;O);0;0; a 


OQ)O,O;Q;0O) OQ jOs;QO;O);0/;0} a 
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Please answer the following questions with regard to your work with students in the AMP after-school 


program. 


Grouping Structure 


1. How often do you use the following grouping structures in the after-school class? 


é Pes than 1-3 times a 1-3 timesa | 4-5 times ply ie 
Grouping Structures Never once a times a 
month week a week 
month day 
Whole class/Large group o o o o o o 
Individual Work o oO o o o oO 
Small groups/Pairs o o o 0 Oo Oo 


Program Participants 


2. Do you feel that the AMP after-school program is appropriate to the reading levels of the students who 


are currently participating? 

oO Notat all appropriate 

O Somewhat appropriate 

O Very appropriate (Skip to Q17) 


16b. Please indicate the proportion of your AMP students for whom the following statements are true. 


ples Most About A few Hardly 
almost all any 
Benes students half students etadlenes 
These students should not be in the AMP 
class because their reading levels are too 0 Oo 0 o o 
high. 
These students should not be in the AMP 
class because their reading levels are too o Oo 0 o o 
low. 


3. Are there students who are not in the after-school program who should be? 


fe) Yes — Please explain: 


fo) No 


4. Please rate the proportion of your students for whom you think the following statements about the 


AMP after-school program are true. 


: AO Most About A few Btly, 
The AMP after-school program is... almost all any 
students half students 
students students 
a. engaging. o oO og og Oo 
b. relevant to their interests. oO o o o oO 
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c. motivating. o o o 0 
d. appropriate to their literacy needs. o o o 0 
e. appropriate to their learning style. o o o o 


Use of AMP Software 


5. How comfortable are you with the after-school Achieving Maximum Potential (AMP) software 


program for struggling readers? 
o Not comfortable 

Oo Somewhat comfortable 

O Moderately comfortable 

o Extremely comfortable 


6. How frequently do you use the Achieving Maximum Potential (AMP) software for struggling readers 


in the after-school program? 

Oo Never 

Less than once a month (Skip to Q21) 
1-3 times a month (Skip to Q21) 

1-3 times a week (Skip to Q21) 

4-5 times a week (Skip to Q21) 
Multiple times a day (Skip to Q21) 


oo0o0°0 


14b. Ifyou said that you “never” use the AMP software, please indicate why by checking the appropriate 


responses below. (Check all that apply) 


The AMP software is not installed on my computers. 


I do not know how to use the AMP software. 


participating 
[1 Other (Please 
specify) 


I do not have computers in the classroom where I teach AMP. 


The computers in the classroom where I teach AMP are not working. 
I do not believe that the AMP software is effective at building students’ literacy skills. 


Instructional Practices 


7. How often do you use the following practices or materials with Tier 3 students in the AMP after-school 


program to help them increase reading comprehension? 


The AMP software is not appropriate to the reading levels of the students who are currently 


Eos ihay fe fae a 
Use of Instructional Practices Never once a < - times a 
month seigntlell aeeuke week 

Explicit instruction in use of summarizing as a comprehension strategy 0 0 im) 0 im) 
Explicit instruction in use of questioning as a comprehension strategy im) 0 0 0 o 
Explicit instruction in use of predicting as a comprehension strategy im) 0 i) 0 im) 
Explicit instruction in using text structure (the organizational 5 q 5 o q 
arrangements used to present information) as a comprehension strategy 
Explicit instruction in use of visualization as a comprehension strategy 0 0 0 o o 
Explicit instruction in use of inferring as a comprehension strategy 0 i) im) im) im) 
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Less than poe pee 4-5 
Use of Instructional Practices Never once a Ry times a 
month sept al lar week 
Explicit instruction in use of metacognition (students select appropriate 
: : ; 0 i) 0 0 0 
comprehension strategies) as a comprehension strategy 
Establishing the purpose for reading. 0 0 0 im) 0 
Monitoring students’ comprehension through questioning. 0 0 im) im) 0 
Making connections to background knowledge. 0 0 0 0 0 
Making connections between texts. 0 0 0 0 0 
Synthesizing information within text or across texts. 0 0 0 0 0 
Using differentiated instruction (using different instructional methods, 
modalities, materials, etc. for different groups or individuals to address 
unique learning needs) 
Use of before, during, and after (BDA) reading strategies for 
comprehension instruction (A student constructed a mental framework 
for reading begun before reading even begins, strengthened as students Oo 0 o o o 
interact with the text during the reading, and reflected upon after 
reading.) 
Use of the gradual release of responsibility model for reading 
comprehension instruction (Leading students from ‘Modeled instruction’ 4 q q 5 qo 
to ‘Shared instruction’ to ‘Guided practice’ and finally ‘Students’ 
independent practice’) 
Using Pairing for Partner Reading & Content Too (PRC2) for 5 q 5 qo qo 
comprehension instruction 
8. How often do you use the following practices or materials with Tier 3 students in the AMP after-school 
program to help them build their vocabulary knowledge? 
Less than fe ie 4-5 
Use of Instructional Practices Never once a “ in times a 
month Ranthe leaweck week 

Explicit instruction in vocabulary 0 0 0 0 0 
Modeling the use of word parts 
Review of vocabulary words 0 0 0 o o 
Use of vocabulary notebooks 0 0 0 o o 
Use of the PRC2 for vocabulary development. 0 0 0 0 o 
Use of before, during, and after (BDA) reading strategies for vocabulary 4 5 5 q q 
instruction 
Use of the gradual release of responsibility model for vocabulary oO oO o oO o 
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9. How often do you use the following practices or materials with Tier 3 students in the AMP after-school 
program to help them develop fluency? 


Tees than ae a a 
Use of Instructional Practices Never once a i e times a 
month week 


Teacher read aloud 


Teacher interactive read aloud 


Shared reading (students and teacher take turns in reading) 


Modeling reading for students 


Explicit instruction in guided oral reading 


Focusing instruction on proper and meaningful phrasing 


Use of the PRC2 for fluency instruction. 


Students listen to audio books, play aways 


O,/QO;0/0O/;0/0;/0;0/0 
QO, O;0/0O/;/0/;/0);/0;0/0 
QO, QO;0/0O/;/0;/0;/0;0/0 
QO; QO;0/0O/;/0;/0);/0;0/0 
QOQ,;QO;0/0O,;0;/0;/0;0/0 


Use of the gradual release of responsibility model for fluency instruction 


10. During your work with Tier 3 students in the AMP after-school program, how often do you use the 
following techniques to help students develop better reading strategies and skills? 


Techniques Never/ Not Less than 1-3 times | 1-3 times | 4-5 times a 
Familiar once a month | a month a week week 
Everybody Reads To (ERT) 0 0 o o o 
Exclusion Brainstorming i) 0 0 o 0 
List-Group-Label 0 0 0 o o 
Predict-Locate-Add-Note (PLAN) 0 i) im) o 0 
ReQuest o o o Oo go 
ene eats ay for Effective Reading 5 q q q q 
ABC Graffiti 0 0 0 0 0 
Ga and Summarizing Procedure q q 5 o q 


Use of Assessment Data 
11. Please indicate the extent to which you use student assessment data for each of the following purposes 
within the AMP after-school program. 


Toa 
Use of Assessment Data Hepat Hoseute Moderate Tos Uarge 
All extent Extent 
Extent 

Differentiating instruction. 0 0 0 0 
Identifying skills that need to be retaught or 5 q q q 
retaught. 

Monitoring student reading progress. 0 im) im) 0 
Creating instructional groups (in-class). 0 i) 0 0 
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12. How often do you meet with ELA classroom teachers at the following grade levels to discuss 
instruction-related issues regarding your work with students in the AMP after-school program?. 


Frequency of Meetings with Grade-Level 
Teachers 
L 1- 
eae ree te a 
Never times | times 
oncea a 
athe Uimaewthllee week | a week 
Grade 6 teachers im) oO ) =) Oo 
Grade 7 teachers oO ) ) =) Oo 
Grade 8 teachers oO ) m) =) a) 
Overall (consider all the teachers that you work with 
regardless of the grade level they teach) eo 2 7 a 7 


13. How often do you meet with SIXTH-GRADE classroom teachers to discuss implementing each of the 


following instructional methods with students in the AMP after-school program? 


How often discussed with classroom teachers 
Instructional methods and activities for re 1-3 1-3 4-5 
than ; ; ' 
AMP program Never times a times a times a 
oncea 
month week week 
month 
a. Differentiated instruction 0 0 0 0 o 
b. Student groupings 0 0 0 0 o 
c. Use of AMP materials 0 0 0 0 0 
d. Using specific AMP and Striving 
Readers instructional techniques for 0 Oo 0 im) o 
comprehension instruction 
e. Using specific AMP and Striving 
Readers instructional techniques for 0 Oo i) 0 o 
vocabulary instruction 
f. Using specific AMP and Striving 
Readers instructional techniques for i) i) 0 0 0 
fluency instruction 
g. Discussing specific students’ reading q 
progress. 
h. Coordinating instruction 
i. Using student assessment data for q 
instructional planning 
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Use of Handheld Computers in the AMPAfter-School Program 


22. a) Do you use handheld computers (Palm Pilots) to teach literacyduringthe AMP class? 


Yes (Skip to Question 26.) 
No 


b) If you are NOT yet using handheld computers during the AMP class, please indicate why you are 
not using them below (Check all that apply) and then SKIP to Question 32: 


The computers have not been made available for the AMP classes. 


The computers and associated software do not integrate well with the AMP program. 
Some or all of the computers are not working properly. 


Some or all of the necessary software applications have not been installed on the computers. 


I have not received sufficient professional development to feel comfortable using them. 


Because the Striving Readers program provides only 10 computers per classroom, and I do not 
like to have some students use them while others can not. 


I do not feel that they offer sufficient added benefit compared to traditional media (e.g. print, 
paper and pencil) to be worth the trouble. 


Other (please specify): 


23. Which grouping strategies do you support through the use of handheld computers (Palm Pilots) 
duringthe AMP class? (Check all that apply) 
O Whole class/ Large group 
O Small group/pairs 
O Individual work 
24. How often do your Tier 3 students use handheld computers (Palm Pilots) duringthe AMP class? 
o _Less than once a month 
o 1-3 times a month 
o 1-3 times a week 
o 4-5 times a week 
25. Rate how comfortable you are withusing the Palm Pilotsto support your literacy instruction during the 


AMP class. 


, 2 3 4 > 
Not at all Very 
Comfortable Comfortable 
oO a) ) oO oO 
26. Which specific academic foci or instructional objectives do you support with the use of handheld 
computers (Palm Pilots) during the AMP class? (Check all that apply) 
[1] Fluency 
[1 Vocabulary development 


Developing students’ reading comprehension strategies 
Writing skills 
Word parts 


Word recognition 


Spelling 
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27. 


Grammar 


Locating information 


Evaluating information 


Synthesizing information 


Organizing information 


To develop students’ self-directed learning 


Teaching students to identify and use the organizational features of expository writing 


To activate students’ prior knowledge 


Which of the following instructional activities and practices do you support with the use of handheld 
computers (Palm Pilots) duringthe AMP class? (Check all that apply) 


[1 Monitoring distribution and completion of assignments 


Assessing students’ literacy skills 


Monitoring students’ progress 


Differentiating instruction 


Teaching comprehension strategies 


Using comprehension techniques 


Guided reading 


Partner reading 


Individual reading 


Book club discussions 
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28. In the table below, please indicate your frequency and comfort of use of each software application when using the handheld computers during the AMP 


class. 


a) Indicate how frequently your Tier 3 students currently use each software application on the Palm Pilots during AMP. 
b) For those that you are using, rate how comfortable you are withusing each software application on the Palm Pilots to support your 
literacy instruction. 


Handheld Computer 
Software Applications 


a) Frequency 


b) IF USING: 
Rate your comfort level 


Not 
Currently 
Using 


Less than 
once a 
month 


1to3 
times a 
month 


1to3 
times a 
week 


4to5 
times a 
week 


1 


Not at all 
Comfortable 


3 


5 


Very 
Comfortable 


CPS ON-Demand (Safari 
Montage). 


mi) 


Q 


QO 


Q 


Q 


Q 


Q 


iKWL 


Freewrite 


PiCo Maps 


Internet Reciprocal Teaching 


Flingit 


Viewpoint 


Sketchy 


MS Word 


MS Excel 


Slideshow to Go 


Cells 


PAAM management software 
application 


GoManage 


QO; O ;O;O;,O;O;/0);0/;0;/0);0;0/;0 


QO; O jOJO;QO;O;,O;/Q;O)/0/;0;0)/0 


QO; OU jOJO;Q;OV,QO;/Q;O)/Q0;0O;0)/0 


QO; OU jOJO;Q;OV,QO;/Q;O);0/;0;0)/0 


QO; O jO;,O;,O;O;/QO);0;/0;/0);0;0/;0 


QO; O jOJO;O;OVQO;/Q;O);Q;0;0)/0 


QO; O jOJO;Q;OVQO;/Q;O);Q/;O0;0)/0 


QO; OU jOJO;Q;O;,QO;/Q;O);Q;/0;0);0; a 


QO; OU jOJO;/Q;OVQO;/Q;O;Q;/Q;0O);0; a 


QO; OU jOJO;Q;O;,Q;/Q;O)QO;0;0);0; a2 


QO; O jOJO;Q;OVQO;/Q;O)/0/;0;0)/0 
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29. For each of the handheld computer software applications listed below, please indicate which literacy instructional objectives are supported by your use of 
that application during your instruction of Tier 3 students during AMP classes. (Please check all that apply.) 


Handheld Computer Software 


Not 
Using 


Instructional Objectives supported by this application 


Vocabulary 
Development 


Fluency 


Reading 
Comprehension 


Writing 
Skills 


Word 
parts 


Word 
recognition 


Spelling 


Grammar 


CPS ON-Demand (Safari 
Montage). 


Oo 


oO 


o 


oO 


im) 


iKWL 


Freewrite 


PiCo Maps 


Internet Reciprocal Teaching 


Flingit 


Viewpoint 


Sketchy 


MS Word 


MS Excel 


Slideshow to Go 


Cells 


OQ)O;Q;Q0/;0;/0/;/0/0;0/;0/;0; a 


Q)QO;/Q/QO0/0/0/0/0/0/0/0 


Q)O,Q;O;/O),O0/0;/0);0/0;0) a 


OQ)/Q;QO/;0;/0/;/0/0)/0;0/0)0 


O)OV,QO;O;/0O)O0/0/;/0)0/0/0 


OQ)/Q;QO/;0)/0/;/0/0)/0;0/0)0 


Q)QO;/QO/0/0/0/0/0/0/0/0 


Q)O,Q;O;/O),O0/O0;/0);/0/0;0) a 


QOQ)OV,Q;/O;/O),O0/0;/0);0/0;0) a 
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Respondent Information 


30. What is your position? O General Education Teacher OBilingual/ELL teacher OSpecial education teacher 0 
LIT 0 Reading Specialist Other (Please Specify: ) 


31. What subject(s) do you teach? (Check all that apply) 

0 All subjects 

2 Literacy/Reading/English language arts 

OMathematics 

Science 

Social studies 

Other: (Please Specify: ) 


32. At which grade level(s) are you teaching reading? this year (2008-09)? (Check all that apply): 
0K010203040506070809010011 012 


33. What is the name of your school? 
[drop down list] 


a) 


34. How many years have you been teaching? [INSERT TEXTBOX] 


35. How many years have you been teaching at this school? [INSERT TEXTBOX] 


36. How many years have you been teaching reading? [INSERT TEXTBOX] 
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1. 


5. 
6. 


Please answer the following questions about the library schedule and access. 


Chicago Public Schools Striving Readers 


Spring 2009 Librarian Survey 
The following is a survey designed to gather your feedback on the essential components of the Striving 
Readers program. It will take you approximately 15 minutes to complete.Results will be reported in the 
aggregate only;we will not use your name or identify individual respondents. Your feedback is extremely 
valuable to the success of this program. If you have questions about this survey, please contact Rebecca 
Swann at rswann@metisassoc.com or 212-425-8833. 


Do you currently work as a full-time or part-time librarian? 


Full-time 


Part-time 


Have you been endorsed as a librarian? 


Yes 
11 No 


Are there any other librarians on staff? 


C1 No 


1 Yes If yes, how many? 


Are there any library aides on staff? 


[11 No 


How long have you been a librarian? —__ 


1 Yes If yes, how many? 


How long have you been a librarian at this school? 


7. a) Please use the tables below to indicate the library hours during the school year . 
Weekday Hours 

Number of hours open each day 
Time of Day during the school year 


Mon. 


Tues. 


Wed. 


Thurs. 


Fri. 


Before school: 


During school: 


After school: 


Evenings (after 5:00): 


Weekend Hours 


Time of Day 


Number of hours open each day 
during the school year 


Sat. 


Sun. 


Daytime Hours: 


Evenings (after 5:00): 


A-235 


b) Students can come to the library: |_| 


Only with their class 
On their own, if they have a pass 


On their own, without a pass 


c) Do students or teachers need to schedule a visit to come to the library... 
....to come as a class? 


[11 Yes 
C1 No 


....for students to come on their own? 
O Yes 

O No 

Not Applicable 


....for teachers to come on their own? 
O Yes 

O No 

Not Applicable 


To what extent do the library resources support the Striving Readers program? 
Not at all 
To a small extent 


To a moderate extent 


To a large extent 


How has your school used the funds provided through the Striving Readers program to 
supplement school library resources?Briefly describe materials and resources acquired through 
these funds in the space provided. 
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8. Please use the table below to describe the nature of your collaboration with the school literacy team, grade level teams, individual teachers and your 
work with students during assigned library time. For each activity listed, 
a) in the first column indicate whether this is part of your responsibilities. 
b) Ifit is, indicate in the remaining columns which groups you collaborate with directly in support of that activity. 


a) b) IF YES: 
Is this part of _ | Check which groups you collaborate with in fulfilling this responsibility: 
Leet : Grade Level ee ee Individual Working directly 
Responsibilities: responsibility’? Teams y teachers with students 
Participating in collaborative decision-making about yes SINo 5 5 q q 
student literacy 
Identify and maintain high quality materials for ves CINo q q q q 
recreational reading 
Maintain a middle-grade area in the library Yes ONo 0 0 0 0 
Refer struggling readers to summer programs that Yes OINo q q q q 
support literacy skills 
Provide students with information about extracurricular 
academic activities (e.g., spelling bee, writing TYes ONo Oo Oo 0 o 
competitions, events) 
Develop summer reading activities for struggling readers Yes CINo 5 q q q 
who do not attend summer school 
Plan and conduct read-alouds OYes ONo 0 Oo 0 Oo 
Plan and conduct literature circles OYes ONo i) 0 0 0 
Plan and conduct book clubs OYes ONo 0 0 0 0 
Direct students to resources tied to the curriculum OYes ONo 0 0 0 0 
Identify resources for class projects Yes ONo Oo oO Oo oO 
Identify resources to supplement classroom lessons Yes ONo Oo Oo Oo Oo 
Teach students how to navigate Internet resources Yes ONo Oo 0 i) 0 
Provide instruction on using print information resources OYes ONo 0 0 0 0 
Provide instruction on using electronic information ves No oq qo oq q 
resources 
Teach students research skills OYes ONo 0 im) 0 0 
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10. For each of the following professional development sessions, please indicate: 
a. Whether you participated, and 
b. If so, how useful the session was in helping you support student learning in language arts 


If YES, how useful was (were) these session(s) in 


De y ye helping you fulfill your responsibilities to the 

participate? ae 9 

Professional Development Sessions Striving Readers program? 
No Vea Not Somewhat | Moderately | Extremely 
Useful Useful Useful Useful 

2008 Summer institute 0 Oo Oo 0 Oo 0 
School-year follow-up institutes im) im) i) 0 0 0 
Saturday seminars Oo Oo 0 Oo o o 
School-based professional development im) 0 0 0 0 0 
Sessions during literacy team meetings q q q q q q 
such as study groups, literacy week, etc. 
Graduate coursesat National- q q q q q q 
LouisUniversity 
Librarian workshops through National- 
Louis University/CPS department of Oo 0 0 0 o o 
libraries 
Technology training: use of handhelds q q q q q q 
and software application 


11. As a librarian, what do you need to better support the literacy needs of middle grade students? 


12. What do you need to improve collaboration with classroom teachers? 
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About You 


13. What is the name of your school? [drop down list] 


BURR 
GRAY ee SSS 


Thank you for your time. 
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CPS Striving Readers 
Non-ELA Content Area Teachers LIST Section 


The following is a survey designed to gather your feedback on the essential components of the Striving 
Readers program. It will take you approximately 30 minutes to complete. Results will be reported in the 
aggregate only; we will not use your name or identify individual respondents. Your feedback is extremely 
valuable to the success of this program. If you have questions about this survey, please contact Rebecca 


Swann at rswann@metisassoc.com or 212-425-8833. 


This year 
Last school year 
Before Striving Readers began 


oo0o0o- 


Comprehensive Instruction 


I do not integrate literacy instruction into my content area. (Skip to Question 6) 


When, if at all, did you start integrating literacy into your content area instruction? 


2. How often do you use the following practices to help students increase reading comprehension? 


a | 43 13 | 45 
Use of Instructional Practices Never times a | timesa | timesa 
once a 
month week week 
month 
Explicit instruction in use of summarizing as a q q 5 q 5 
comprehension strategy 
Explicit instruction in use of questioning as a comprehension oO oO oO oO 
strategy 
Explicit instruction in use of predicting as a comprehension q q q qo q 
Strategy 
Explicit instruction in using text structure (the 
organizational arrangements used to present information) as a im) 0 0 0 0 
comprehension strategy 
Explicit instruction in use of visualization as a 
. 0 i) i) 
comprehension strategy 
Explicit instruction in use of inferring as a comprehension 4 q q qo 5 
strategy 
Explicit instruction in use of metacognition (students select 
appropriate comprehension strategies) as a comprehension im) 0 im) im) 0 
strategy 
Establishing the purpose for reading. o 0 0 o 0 
Monitoring students’ comprehension through questioning. 0 0 0 im) 0 
Making connections to background knowledge. im) i) i) im) 0 
Making connections between texts. im) 0 i) im) 0 
Synthesizing information within text or across texts. im) 0 0 0 i) 
Using differentiated instruction (using different instructional 
methods, modalities, materials, etc. for different groups or 0 0 i) im) 0 
individuals to address unique learning needs) 
Use of before, during, and after (BDA) reading strategies for 
comprehension instruction (A student constructed mental 
framework for reading begun before reading even begins, im) 0 0 0 0 
strengthened as students interact with the text during the 
reading, and reflected upon after reading.) 
Use of the gradual release of responsibility model for reading 5 q q 5 5 


comprehension instruction (Leading students from ‘Modeled 
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finan 1-3 1-3 4-5 
Use of Instructional Practices Never alneea times a | timesa | timesa 
month month week week 
instruction’ to ‘Shared instruction’ to ‘Guided practice’ and 
finally ‘Students’ independent practice’) 
Using Pairing for Partner Reading & Content Too (PRC2) for 4 q q 5 q 
comprehension instruction 
3. How often do you use the following practices to help students build their vocabulary knowledge? 
tian | times | 23 | 45 
Use of Instructional Practices Never Coew - times a | times a 
month | month meee bidet 
Explicit instruction in vocabulary Oo 0 im) 0 0 
Modeling the use of word parts 
Review of vocabulary words 0 0 0 0 im) 
Use of vocabulary notebooks 0 0 0 o o 
Use of the PRC2 for vocabulary development. i) i) 0 0 0 
Use of before, during, and after (BDA) reading strategies for q 5 q qo q 
vocabulary instruction 
Use of the gradual release of responsibility model for q q q qo q 
vocabulary instruction 
4. How often do you use the following practices to help students develop fluency? 
than, | ones | ote) es 
Use of Instructional Practices Never ances f times a | times a 
month | month Nee week 
Teacher read aloud i) 0 0 im) 0 
Teacher interactive read aloud 0 0 0 o 0 
Shared reading (students and teacher take turns in reading) og oO Oo Oo Oo 
Modeling reading for students 0 0 0 0 o 
Explicit instruction in guided oral reading 0 i) 0 im) 0 
Focusing instruction on proper and meaningful phrasing i) 0 0 o 0 
Use of the PRC2 for fluency instruction. 0 0 0 0 o 
Students listen to audio books, play aways 0 0 0 o o 
Use of the gradual release of responsibility model for fluency 5 5 q q q 


instruction 
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5. How often do you use the following techniques to help students develop better reading strategies and 


skills? 
Techniques Never/ Not Less than 1-3 times | 1-3 times | 4-5 times a 
Familiar once a month | amonth a week week 
Everybody Reads To (ERT) 0 Oo 0 0 0 
Exclusion Brainstorming im) i) 0 0 o 
List-Group-Label 0 0 0 o o 
Predict-Locate-Add-Note (PLAN) 0 i) 0 0 0 
ReQuest o o o oO oO 
ai atee ANGE pis for Effective Reading q 4 q q q 
Oo Oo q q 0 
ABC Graffiti 0 0 0 0 o 
Ca and Summarizing Procedure 5 5 5 5 5 
6. Please indicate how often (if at all) you have met with each of the following literacy experts 
during the current school year. 

oe La 13 | 45 

Meetings with Literacy Experts (Do not | Never ae times a | timesa | timesa 
have) once a | month | week week 
month 

Literacy Intervention Teacher 0 0 Oo Oo Oo 

Lead Literacy Teacher/Literacy Coach Oo 0 0 0 0 0 

Literacy Team Members 0 Oo qo im) q 

a a Coordinator (district- q q q q q 

Regular ELA teachers i) 0 0 0 0 

Other (specify): Oo 0 Oo o o 


7. Please indicate how often (if at all) you discussed each of the following topics with any of the 
above literacy experts during the current school year. 


Less than | 1 3 times | 1-3 times | 4-5 times 
Discussion Topics with a Literacy Expert Never once a 
a month a week a week 
month 
a. Differentiated instruction i) Oo 0 Oo Oo 
Student groupings Oo 0 i) 0 0 
Use of Striving Readers text sets and text 4 q q 5 q 
sets teacher guides 
d. Use of technology resources: desktop 
computers, handhelds- palms, LCD o Oo Oo 0 Oo 
projector, etc. 
e. Use of PRC2 instructional framework 
Using specific Striving Readers instructional 7 q 5 q q 
techniques for comprehension instruction 
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Less than | 1 3 times | 1-3 times | 4-5 times 
Discussion Topics with a Literacy Expert Never once a a mnisatins |, ca seteke sietale 
month 
g. Using specific Striving Readers instructional 7 5 q q 5 
techniques for vocabulary instruction 
h. Using specific Striving Readers instructional 
techniques for fluency instruction 
i. Specific students’ reading progress. 
j. Using student assessment data for 
instructional planning 
Grouping Structure 
8. How often do you use the following grouping structures in your classes? 
Less than é : ¢ 
: 1-3 times a 1-3 times a | 4-5 times 
Grouping Structures Never once a 
month week a week 
month 
Whole class/Large group o 0 0 0 0 
Individual work o o im) 0 i) 
Small groups 0 0 0 Oo 0 
Partner Reading o o 0 0 0 
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School-wide Intervention Materials/Hide Qs 9-11 if response to QI = “I do not integrate literacy”] 


9. For the following school-wide intervention materials: 
= Indicate how frequently you currently use the materials to teach literacy through your subject area. 
= For those that you are using, rate how comfortable you are with using these materials to support student learning in language arts. 


b) IF USED: 


a) Frequenc 
abe 2 Rate your comfort level 


Materials N/A Not Less than| 1 to3 1 to3 4to5 1 5 
(Do Not | Currently | oncea times a times a times a Not at all 2 3 4 Very 
Have) Using month month week week [| Comfortable Comfortable 
Text sets — Text sets q 7 7 7 q 5 5 5 5 5 5 


teacher guides 


Media centers (three q q q q q q q q q q 4 
computers and a printer) 


Listening centers 
(Classroom CD or cassette 


player, read-along audio 0 im) 0 0 0 0 0 0 0 0 0 
books, playaways and 

headphones) 

Handheld computers (Palm q q q q q q q q q q q 
Pilots) 

Classroom library 0 i) i) 0 0 i) 0 0 0 0 0 
School library resources 0 0 0 0 0 0 0 o o o o 
Vocabulary notebooks 0 0 0 0 0 0 0 o o o o 
Textbooks i) 0 0 0 0 0 0 0 0 0 0 
Reading response notebooks 0 0 0 0 0 0 0 0 o o o 
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10. For each of the materials listed below, please indicate which grouping strategies are supported by your use of that 
material in your classroom when teaching literacy through your content area. (Check all that apply) 


Not Grouping Strategies 
Materials Using Whole Class/ Small Group/ Individual 
Large Group Pairs Work 
Listening centers 0 0 im Oo 
Media centers 0 im) Oo im) 
Text sets and Text sets teacher guides 0 0 0 0 
Handheld computers (Palm Pilots) 0 0 0 0 
Classroom library 0 Oo oq fm) 
Vocabulary notebooks 0 0 0 0 
Textbooks =) Oo o fm) 
Reading response notebooks 0 0 im o 


Use of Text Sets and Text Sets Teacher Guides 


11. a) Do you use text sets (informational reading and listening materials at different levels and with varied text structures 
and organizational features that offer perspectives on a theme) to teach reading or literacy? 


Yes (SKIP to Question 12) 
No 


b) Ifyou are NOT yet using text sets, please indicate why you are not using them below (Check all that apply) 


They have not been made available to me 


The reading level is not appropriate for my students. 


The content is not relevant/interesting to my students 


I did not receive text sets for the topics that we are covering in my class 


I have texts sets, but not the text sets teacher guides 
Other (specify): Professional Development in Literacy Instructional Practices 


12. For each of the following literacy teaching practices, indicate: 
a. Did you receive professional development through the Striving Readers program during the current 
school year? 
If so: 
b. Please indicate whether you are using this practice as part of your content area instruction, and 
c. Rate your comfort implementing each teaching practice within your content area instruction. 
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IF YES: 
a) : 
Received | 5) Using as part 
PD? of content c) Rate your comfort level 
Literacy based teaching practices instruction? 
1 2 3 4 5 
No | Yes No Yes Not at all Very 
Comfortable Comfortable 

Building academic vocabulary 0 o 0 0 o Oo 0 0 0 
Using classroom libraries 0 o 0 0 o o o o 0 
Creating literacy-rich classroom environments i) im) 0 Oo o 0 0 0 0 
Differentiating instruction 0 0 0 0 0 0 0 0 0 
Direct vocabulary instruction o o 0 o o o 0 0 0 
Incorporating text sets in your instruction 0 o 0 0 o 0 0 0 i) 
Increasing student motivation 0 0 0 0 0 0 0 0 0 
Supporting students’ self-directed learning o o o o o o 0 0 0 
Using before, during, and after reading strategies 0 im) 0 i) im) 0 0 0 0 
Using formal assessments to guide instruction 0 0 0 0 o o o o o 
Using informal assessments to guide instruction 0 o 0 0 o o 0 o 0 
Using handheld computers (Palm Pilots) i) im) mt) 0 o 0 0 0 0 
Using literacy-based software 0 o 0 0 o o o o o 
Using the PRC2 model o o o o o o 0 0 0 
Using the whole-part-whole classroom instruction model 0 o 0 0 o o o o 0 
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13. Please check the techniques in the list below for which you would like to receive more training. 
(Check all that apply) 


Marzano’s vocabulary 

KWL 

Word Study/word sorting 

Using PRC2 for fluency instruction. 

Using PRC2 for comprehension instruction. 

Using PRC2 for vocabulary development. 

Everybody Reads To (ERT) 

Exclusion Brainstorming 

List-Group-Label 

Predict-Locate-Add-Note (PLAN) 

ReQuest 

Interactive Notation System for Effective Reading and Thinking (INSERT) 
Read Aloud/Think Aloud 

ABC Graffiti 

Guided Reading and Summarizing Procedure (GRASP) 
Teachingsummarizing as a comprehension strategy 
Teachingquestioning as a comprehension strategy 
Teachingpredicting as a comprehension strategy 
Teachingtext structure as a comprehension strategy 
Teachingvisualization as a comprehension strategy 


Teachinginferring as a comprehension strategy 


OUOUOOOUVOOVKCOOOOVCOCOOOOCULCOD 


Teachingmetacognitionas a comprehension strategy 
Respondent Information 
14. At which grade level(s) are you teaching this year (2008-09)? (Check all that apply): 


0kK010203040506070809010011 012 


15. What is the name of your school? [drop down list] 


BURR 
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FISKE SMYTH, J 
FULLER TALCOTT 
GALE COM TELPOCHCALLI 


GOMPERS VOLTA 
GRAY 


16. How many years have you been teaching? [INSERT TEXTBOX] 


17. How many years have you been teaching at this school? [INSERT TEXTBOX] 


18. How many years have you been teaching in your subject area? [INSERT TEXTBOX] 


Teachers will be reimbursed by CPS-Striving Readers for their time to complete this survey. In order to be 


reimbursed we need you to identify yourself so that we can verify that you completed the survey. If you would like 
to be reimbursed, please provide your name and email address below, and be sure that you identified your school in 
the previous item. Your survey responses will still remain strictly confidential and will never be reported in any 
form that would allow anyone to connect your responses with your name. Providing this information is optional. 


Your Name: Email: 


Thank you for completing this survey! 
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Chicago Public Schools (CPS) Striving Readers 
Spring 2009 Literacy Improvement Survey for Teachers — Control Schools 


The following is a survey designed to gather your feedback on the essential components of your school’s literacy 
program. Survey results will be reported in the aggregate only. We will not use your name or identify individual 
respondents. Your feedback is extremely valuable to the success of this program. If you have questions about this 
survey, please contact Rebecca Swann at rswann@metisassoc.com or 212-425-8833. 


1. What is your primary role or teaching assignment? 


(Select the single best option. 
- If you teach English language arts as well as other subject area(s), please respond to the survey with your 
role as an ELA teacher in mind.) 


General education teacher (self-contained classroom teacher) (Continue with LIST Q3) 


English language arts teacher 
Teach English language arts and other academic subject eS 
Teach other academic subjects but notEnglish language arts 
Bilingual/ELL teacher 
Special education teacher 
Librarian 
Reading specialist 
Other (please specify): 

Does this role include teaching of English language arts? 


e Yes 
e No 


2. What other subjects do you teach? 


Oo Math 

O Science 

oO Social Studies/Humanities 
O Other (please specify) 
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Comprehensive Instruction 


3. How often do you use the following practices to help students increase reading comprehension? 


Less 1-3 1-3 

Not than | times | times a 

Use of Instructional Practices or Never times a 
Familiar once a a a eck 
month | month | week sil 

Explicit instruction in use of summarizing as a q q 5 5 5 q 
comprehension strategy 
Explicit instruction in use of questioning as a q q q qq oq oO 
comprehension strategy 
Explicit instruction in use of predicting as a 5 q q 5 q 5 
comprehension strategy 
Explicit instruction in using text structure (the 
organizational arrangements used to present information) i) 0 i) im) 0 0 
as a comprehension strategy 
Explicit instruction in use of visualization as a q 5 q 5 5 + 
comprehension strategy 
Explicit instruction in use of inferring as a comprehension q o q q 
strategy 
Explicit instruction in use of metacognition (students 
select appropriate comprehension strategies) as a 0 o 0 0 o Oo 
comprehension strategy 
Establishing the purpose for reading. 0 o 0 0 o o 
Monitoring students’ comprehension through questioning. i) im) 0 0 im) 0 
Making connections to background knowledge. 0 o 0 0 o Oo 
Making connections between texts. 0 o 0 0 o o 
Synthesizing information within text or across texts. 0 im) i) 0 im) 0 
Using differentiated instruction (using different 
instructional methods, modalities, materials, etc. for 
different groups or individuals to address unique learning m o ? 2 
needs) 
Use of before, during, and after (BDA) reading strategies 
for comprehension instruction (A student constructed 
mental framework for reading begun before reading even 0 o 0 0 o Oo 
begins, strengthened as students interact with the text 
during the reading, and reflected upon after reading.) 
Use of the gradual release of responsibility model for 
reading comprehension instruction (Leading students from q qo q q o q 
‘Modeled instruction’ to ‘Shared instruction’ to “Guided 
practice’ and finally ‘Students’ independent practice’) 
Using partner reading to enhance comprehension o o 0 ) o Oo 
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Less 1-3 1-3 
Not than | tim tim a 
Use of Instructional Practices ae Never ses mes | times a 
Familiar once a a a k 
month | month | week hua 
instruction 
4. How often do you use the following practices to help students build their vocabulary knowledge? 
Less 1-3 1-3 4-5 
Use of Instructional Practices Not Never tian ante || ES | Manes 
Familiar once a a a a 
month | month | week | week 
Explicit instruction in vocabulary 0 0 im) o 0 0 
Modeling the use of word parts 0 im) im) o 0 0 
Review of vocabulary words im) o o o 0 im) 
Use of vocabulary notebooks o o o o 0 oO 
Use of partner reading to enhancevocabulary Oo o o o o oO 
Use of before, during, and after (BDA) reading q q qo qo q q 
strategies for vocabulary instruction 
Use of the gradual release of responsibility model for 4 qo qo qo q q 
vocabulary instruction 
5. How often do you use the following practices to help students develop fluency? 
Less 1-3 1-3 
Not than | tim ti ar 
Use of Instructional Practices oe Never nas mes | times a 
Familiar once a a a eek 
month | month | week bi 
Teacher read aloud 0 0 o 0 0 0 
Teacher interactive read aloud 0 o 0 0 0 o 
shared reading (students and teacher take turns in q o q q q q 
reading) 
Modeling reading for students 0 0 o o o o 
Explicit instruction in guided oral reading 0 0 o 0 0 0 
Focusing instruction on proper and meaningful phrasing 0 0 o 0 o 0 
Use of partner reading to enhance fluency instruction. 0 0 o 0 0 0 
Students listen to audio books, play aways 0 o o o o o 
Use of the gradual release of responsibility model for q qo o q o q 


fluency instruction 


A-251 


6. How often do you use the following techniques to help students develop better reading strategies and skills? 


Techni Not Neves Less than 1-3 times | 1-3 times | 4-5 times a 
iene lay Familiar ie once a month | a month a week week 
Everybody Reads To (ERT) 0 0 o 0 o o 
Exclusion Brainstorming 0 0 im) 0 0 0 
List-Group-Label 0 0 im) 0 0 0 
Predict-Locate-Add-Note (PLAN) i) 0 0 0 0 0 
ReQuest Oo o o Oo oO Oo 
Interactive Notation System for Effective Reading 
and Thinking (INSERT) S ey 2° 2 z = 
ABC Graffiti i) 0 im) 0 0 0 
Guided Reading and Summarizing Procedure 
(GRASP) 0 0 o 0 0 0 
7. How often do you use the following grouping structures in your classes? 
Less than | 13 timesa | 1-3timesa | 4-5times | Multiple 
Grouping Structures Never once a times a 
month week a week 
month day 

Whole class/Large group oO o o Oo o Oo 

Individual Work Oo Oo oO o Oo Oo 

Small groups or Pairs o oO o o oO oO 


8. Considering your own instruction (not that of other instructors in your classroom), how often do you apply 
differentiated instruction (using different instructional methods, modalities, materials, etc. for different groups 
or for different individuals within groups, to address the unique learning needs of different students) in your 


classroom? 
Almost 
‘ About half Most of every 
Never Rarely Pecasionally the time the time lesson or 
activity 
| | Oo Oo oO | 
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Purposeful Assessment 


9. Indicate how you use the data from the following assessments. (Please check all that apply.) 


Progress Assess 


Assessments Not Using Screening Diagnostic Benchmarking Monitoring Guieonies 


Reading Benchmark 7 oO o o o o 
Assessment 


Illinois Standards Achievement 
Test 


Basic Reading Inventory (BRI) 


Informal assessments 


Fluency Snapshots 


Spelling Inventories 


Other: 


Other: 


QOQ;Q);Q0,;Q0;0;0);0; QO 
O;O;O/;0;0;0/;0; QO 
QOQ;)QO;Q0O,;0/;0;0);0; OQ 
OQ;Q);QO,;Q0;0;0);0; QO 
O;O;0O/;0/;0;0/;0; QO 
QOQ;Q;Q0,;Q0;0;0);0; QO 


Other: 


Data-Driven Instruction 


10. Please indicate the extent to which you use student assessment data for each of the following purposes. 


Use of Data Not at All To Some extent Be Pas To a Large Extent 


Placing students in intervention 
programs. 


Differentiating instruction. 


Identifying skills that need to be taught 
orretaught. 


Monitoring student reading progress. 


QO};O);) Q |;QO; Oa 
QOQ);O);) Q ;QO; Oa 


O);QO;); OQ |/O 
QO;QO; Q /Q 


Creating instructional groups (in-class). 


Grade-Level Teams 
11. Do you currently have grade-level (horizontal) teams at your school? 


Oo Yes 
Oo No(lIfno, skip to Question 15) 
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12. Overall, rate the grade-level team’s ability to use classroom assessment data in the following ways. 


Use of Data 


Poor 


Fair 


Good 


Excellent 


Not Sure 


Address the literacy needs of all students. 


o 


o 


0 


mi) 


Oo 


Address the needs of struggling readers. 


Formalize lesson plans. 


Identify students who are eligible for targeted 
interventions. 


Identify strengths. 


Identify teaching and learning strategies. 


Improve classroom practice. 


O;O;O) OQ ;O;O 


QOQ/;/O);O,;, Q ;QO/;0 


QOQ)/O);QO,;, Q ;QO/0 


Q)/O);O);) Q ;O/;0 


O;QO;0O);) Q ;O;O 


Literacy Teams 


13. Do you currently have a (vertical) literacy team in place at your school? 


Oo Yes 
0 No (lIfno, skip to Question 17) 


14. Overall, rate the quality of the literacy team’s performance in the following areas. 


Performance Areas 


Poor 


Fair 


Good 


Excellent 


Not Sure 


Using assessment data to pinpoint the staff's 
professional development needs. 


0 


im) 


im 


0 


Oo 


Addressing the needs of all students. 


Addressing the needs of struggling readers. 


Addressing the needs of grade-level teams. 


Improving literacy instruction at your school. 


QO;QO;O);0 


O}O)O);0 


QO;Q/;O);0 


QO;Q/;O/;0 


O;O)QO);0 
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School-wide Intervention Materials 


15. For each of the materials listed below, 


Indicate how frequently you currently use the materials to teach literacy. 


For those that you are using, rate how comfortable you are withusing these materials to support student learning in language arts. 


Materials 


a) Frequency 


b) IF USED: 


Rate your comfort level 


N/A 


Not Less than; 1to3 1 to3 4to5 1 2 3 4 5 
ae aut Currently | oncea times a times a times a Not at all Very 
pl Using month month week week Comfortable Comfortable 
Working) 
Listening center (Classroom CD 
orcassette player, read-along audio 0 im) 0 i) 0 0 i) 0 0 0 0 
books, Playaways and headphones) 
Media center (classroom-based 
station with three computers and a 0 im) i) im) i) 0 0 0 0 0 0 
printer) 
Text sets ( content related books of 
different reading levels, genres and 0 im) i) i) 0 0 0 0 0 0 0 
subject themes) 
Classroom library 0 0 0 0 0 0 i) 0 0 0 0 
Vocabulary notebooks 0 o 0 0 0 o 0 o Oo o o 
Textbooks 0 0 i) i) 0 0 0 i) 0 0 0 
Reading response notebooks 0 o 0 0 0 0 0 0 o o o 
School library 0 im) 0 i) i) im) i) 0 0 0 0 
Other informational texts (other q oq 4 q q oq q q 4 q q 


than text sets) 
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16. For each of the materials listed across the top of the chart below, please indicate which literacy instructional goals are supported by your use of that material 


Media 
centers 
L 


in your classroom. (check all that apply.) 


Instructional goals that each material 
is used to support: 


Spelling 


Listening 


centers 


4 


tit 


OO 


Grammar rr §B Ea 


To teach content themes 


Q 


Materials 


Classroom 
Text sets library 
J L 


OVO 


Q 


| ea 

notebooks Textbooks 

J 1 

o o 

o o 
o 


Oo 


oe 


To develop students’ self-directed i) o 
learning 


To supplement students’ textbook 0 
reading 


Teaching students to identify and use 


text structure 


Teaching students to identify and use 
the organizational features of 
expository writing 


E 


Q 


To activate students’ priorknowledge | OO | OF | OG | GO | GO FT Oo 
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17. For each of the materials listed below, please indicate which literacy instructional practices are 
supported by your use of that material in your classroom. (Please check all that apply.) 


Instructional Practices 
Differentiating 
Materials ~ * Guided Partner Individual Book club Dtterennaine Hg acden oF 
Using : : : . : instruction for English language 
reading reading reading discussions : : 
struggling readers learners/special 
education students 
Listening centers 0 Oo 0 0 0 =) im) 
Media centers 0 0 0 Oo 0 Oo fm) 
Text sets 0 0 0 oO 0 qo o 
Classroom library 0 0 0 0 0 im) Oo 
Vocabulary notebooks 0 Oo 0 0 0 Oo Oo 
Textbooks 0 Oo 0 oO 0 oO fm) 
Reading response 0 0 0 0 0 Oo fm 
notebooks 


18. For each of the materials listed below, please indicate which grouping strategies are supported by your 


use of that material in your classroom. (Check all that apply) 


Not Grouping Strategies 
Matorls Using | Whole Class/ | Small Group/ Individual 
Large Group Pairs Work 
Listening centers o o qo q 
Media centers 0 o q oq 
Text sets o o qo q 
Classroom library 0 0 0 o 
Vocabulary notebooks 0 0 Oo fm) 
Textbooks 0 0 Oo o 
Reading response notebooks 0 0 o Oo 
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Use of Classroom Computers for Literacy Instruction 


19. a) Do you use classroom computers (desktop, laptop, or handheld computers) to teach literacy? 


Yes (Skip to Question20.) 
No 


b) If you are NOT yet using computers to teach literacy, please indicate why you are not using them 
below (Check all that apply) and then SKIP to Question 28: 


I do not have computers available for my students to use. 

I have computers but some or all of them are not working properly. 

Some or all of the necessary software applications have not been installed on the computers. 
I have not received sufficient professional development to feel comfortable using them. 


There are not enough computers for every student and I do not like to have some students use 
them while others can not. 


I do not feel that they offer sufficient added benefit compared to traditional media (e.g. print, 
paper and pencil) to be worth the trouble. 


Other (please specify): 


20. Which grouping strategies do you support through the use of classroom computers during literacy 
lessons? (Check all that apply) 


oO 
O 
O 


Whole class/ Large group 
Small group/pairs 
Individual work 


21. How often do students use classroom computers during literacy instruction in your classroom? 


o Less than once a month 

o 1-3 times a month 

o 1-3 times a week 

o 4-5 times a week 

22. Rate how comfortable you are withusing classroom computers to support your literacy instruction 

1 5 

Not at all Very 

Comfortable Comfortable 

o o 
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23. 


24. 


Which specific academic foci or instructional objectives do you support with the use of classroom 
computers ? (Check all that apply) 


Fluency 


Vocabulary development 


Developing students’ reading comprehension strategies 
Writing skills 
Word parts 


Word recognition 


Spelling 


Grammar 


Locating information 


Evaluating information 


Synthesizing information 


Organizing information 


To teach content themes 


To develop students’ self-directed learning 


Teaching students to identify and use the organizational features of expository writing 


To activate students’ prior knowledge 


Which of the following instructional activities and practices do you support with the use of classroom 
computers ? (Check all that apply) 


Monitoring distribution and completion of assignments 


Assessing students’ literacy skills 


Monitoring students’ progress 


Differentiating instructionfor struggling readers 


Differentiating instruction for English language learners/special education students 


Guided reading 


Partner reading 


Individual reading 


Book club discussions 
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Classroom Library 


25. Please check the ways that you use your classroom libraries. (Check all that apply) 


For content area instruction 


For independent reading 


For small group instruction 


C1 For read alouds 


I do not have a classroom library 


26. Do you use interest inventories to help students self select reading material? 
o Yes 
o No 


27. Do you use interest inventories to guide your purchases for the classroom library? 
o Yes 
o No 


28. Please indicate how true each of the following statements areabout the organization of books in your classroom 


library. 

Not At Slightly Somewhat Very 
My classroom library... All True True True True 
...1S easily accessible to students. 0 0 o o 
...4s well organized and in good shape. i) 0 Oo 0 
...has a checkout system in place. i) 0 0 0 
...includes a variety of reading materials that are q q q q 
appropriate for readers of differing abilities. 
.. includes a variety of texts that appeal to readers with q oq q q 
differing interests. 
...has reading materials grouped by genre. =) Oo oO Oo 
...has reading materials clearly labeled. ) a) oO oO 
..-has both nonfiction and fiction books. oO oO oO oO 


School Library 


29. How often do you take your class to the library? 
Oo Never 

Rarely (less than once a month) 

Sometimes (at least once a month) 

Often (at least once a week) 

Almost daily or daily 


30. To what extent do the library resources support your school’s literacy program? 
Oo Notat all 

To a small extent 

To a moderate extent 

To a large extent 

Don’t know 


ooo0o°o 
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31. How does the librarian work with you? (Check all that apply.) 


Oo 


O 
O 
O 


The librarian does not work with me. 
The librarian provides resources for class projects. 
The librarian and I collaborate on how to supplement lessons with library resources. 


Other (please specify): 


32. To what extent does the librarian consult withclassroom teachers to order reading materials that are 
grade level and content appropriate? 


O 


ooo0o°o 


Not at all 

To a small extent 

To a moderate extent 
To a large extent 
Don’t know 


33. To what extent does the librarianconsider students’ needs and reading abilitieswhen ordering books 
and other reading material? 


Oo 


oo0°o 


Not at all 

To a small extent 

To a moderate extent 
To a large extent 
Don’t know 


34. To what extent does the librarianconsider students’ interests and motivationwhen ordering books and 
other reading material? 


Oo 


ooo0o°o 


Not at all 

To a small extent 

To a moderate extent 
To a large extent 
Don’t know 


35. How does the librarian work with your students? (Check all that apply.) 


Oo 


oO 
oO 
oO 
oO 


Ooo0°0 


Does not work with my students. 

Works with students on research skills. 

Directs students to resources tied to curriculum. 

Conducts read-alouds. 

Provides students with information about extracurricular academic activities (e.g., spelling bee, writing 
competitions, events). 

Assists students with class projects. 

Teaches students how to navigate Internet resources. 

Guides struggling readers to summer programs. 

Other (please specify): 


Collaboration with Literacy Support Staff 


36. Does your school have a literacy enrichment specialist such as a Reading Specialist or Literacy Coach? 


Yes 
Please indicate this person’s title: 
No (Skip to Question 38) 


A-261 


37. How often do you meet or collaborate with the literacy enrichment specialist in the following settings? 


Groiuiis Stmictines Nivex Less than 1-3 times 1-3 times a 4-5 times 
ping once a month | a month week a week 

Scheduled one-on-one meetings o o 0 0 0 
Impromptu one-on-one meetings 
(during lunch, prep periods, o o 0 0 o 
before/after school, etc.) 
Grade-level (horizontal) team qo qo q q + 
meetings 
Literacy (vertical and horizontal) q q q 
team meetings 


38. To what extent has your collaboration with the literacy enrichment specialist facilitated your efforts to use the 
following methods to support struggling readers in your class? 


Extent to which collaboration with the literacy enrichment 
‘ specialist facilitated use of these methods 
Instructional methods 
To asmall To a moderate To a large 
Not at all 
extent extent extent 
a. Differentiating instruction im) 0 0 o 
b. Scaffolding of instruction 0 i) 0 o 
c. Student groupings 0 0 o o 
d. Using the media center o 0 o o 
e. Using listening centers im) i) 0 0 
f. Using classroom computers o 0 0 o 
g. Using text sets o 0 0 o 
h. Using assessment data to monitor student progress 0 0 im) o 
i. Using student assessment data for instructional qo q q q 
planning 


39. To what extent has your collaboration with the literacy enrichment specialist facilitated your ability to provide 
effective instruction in the following areasfor struggling readers? 


Extent to which collaboration withthe literacy enrichment 
F specialist facilitated effective instruction 
Academic areas 
To asmall To a moderate To a large 
Not at all 
extent extent extent 
a. Comprehension 0 0 o o 
b. Fluency 0 0 Oo o 
c. Vocabulary 0 0 o o 
d. Writing skills 0 i) 0 o 
e. Word parts 0 0 Oo o 
f. Word recognition 0 0 o o 
g. Spelling 0 0 o o 
h. Reading/literacy in content areas 0 0 0 o 
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40. Overall, how effective has the the literacy enrichment specialist been in improving the reading skills of struggling readers in your classroom? 


oO Notat all effective 
Minimally effective 
Somewhat effective 
Effective 

Very effective 


Ooo0°0 


Professional Development 


41. For each of the following topics, indicate: 


= Whether you received professional development addressing this topic during the current year 


= Ifso, rate the impact that professional development you received has had on your comfort with each teaching practice. 


If YES, what impact did the professional development have on your 


Receveae De comfort with each teaching practice? 
Teaching practices 
No Yes No Impact Slight Impact anions Major Impact 

Building academic vocabulary 0 im) im) 0 o 0 
Classroom libraries 0 im) im) 0 im) 0 
Creating literacy-rich classroom environments i) im) im) 0 0 0 
Differentiating instruction i) im) im) i) im) 0 
Direct vocabulary instruction i) 0 im) 0 o 0 
Incorporating text sets in your instruction im) 0 im) 0 im) 0 
Increasing student motivation 0 im) im) 0 o 0 
Supporting students’ self-directed learning 0 o o 0 o o 
Using before, during, and after reading strategies 0 im) im) 0 o 0 
Using student assessments to guide instruction 0 o o 0 o o 
Using classroom computers 0 o o 0 o im) 
Using literacy-based software 0 o o 0 o o 
Using partner-reading 0 o o o o o 
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Struggling Readers: Extended Day (Afterschool) Intervention 


42. Does your school currently offer afterschool programming specifically targeting struggling readers? 
Oo Yes 
Oo No (Ifno, skip to Question 42.) 


43. How many of your current students are involved in the afterschool program? 
None (If none, skip to Question 42.) 

1 to 3 

4to6 

7to9 

10 or more 


oooo0o°o 


44. Overall, how effective has the afterschool component been in improving the literacy abilities of struggling 
readers? 
o Notat all effective 


oO Minimally effective 

Oo Somewhat effective 

0 Effective 

oO Very effective 

Oo Don’t Know 
About You [All types of respondents] 
45. What is the name of your school? 
[drop down list] 


46. At which grade level(s) are you teaching reading/English language arts this year (2008-09)? (Check 
all that apply): 


0K 010203040506070809 010011 012 


47. In which of the following settings do you teach literacy? (Check all that apply) 
oO. Self-contained 
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Subject-Area specialist 
Departmentalized 
Double block 

Other (Please specify): 


ooo0o°0 


48. How many years have you been teaching? [INSERT TEXTBOX] 
49. How many years have you been teaching at this school? [INSERT TEXTBOX] 


50. How many years have you been teaching reading? [INSERT TEXTBOX] 


Teachers will be reimbursed by CPS-Striving Readers for their time to complete this survey. In order to be 


reimbursed we need you to identify yourself so that we can verify that you completed the survey. If you would like 
to be reimbursed, please provide your name and email address below, and be sure that you identified your school in 
the previous item. Your survey responses will still remain strictly confidential and will never be reported in any 
form that would allow anyone to connect your responses with your name. Providing this information is optional. 


Your Name: Email: 


Thank you for completing this survey! 
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Chicago Public Schools Striving Readers 
Spring 2009 Librarian Survey — Control Schools 


The following is a survey designed to gather your feedback on the essential components of your school’s 
literacy program. It will take you approximately 15 minutes to complete.Results will be reported in the 
aggregate only;we will not use your name or identify individual respondents. Your feedback is extremely 
valuable to the success of this program. If you have questions about this survey, please contact Rebecca 
Swann at rswann@metisassoc.com or 212-425-8833. 
Do you currently work as a full-time or part-time librarian? 


1. 


5. 
6. 


Please answer the following questions about the library schedule and access. 


Full-time 


Part-time 


Have you been endorsed as a librarian? 


Yes 
11 No 


Are there any other librarians on staff? 


C1 No 


1 Yes If yes, how many? 


Are there any library aides on staff? 


[11 No 


How long have you been a librarian? —__ 


1 Yes If yes, how many? 


How long have you been a librarian at this school? 


7. a) Please use the tables below to indicate the library hours during the school year . 
Weekday Hours 

Number of hours open each day 
Time of Day during the school year 


Mon. 


Tues. 


Wed. 


Thurs. 


Fri. 


Before school: 


During school: 


After school: 


Evenings (after 5:00): 


Weekend Hours 


Time of Day 


Number of hours open each day 
during the school year 


Sat. 


Sun. 


Daytime Hours: 


Evenings (after 5:00): 
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b) Students can come to the library: |_| 
Only with their class 


On their own, if they have a pass 


On their own, without a pass 


c) Do students or teachers need to schedule a visit to come to the library... 
....to come as a class? 


[11 Yes 
C1 No 


....for students to come on their own? 
O Yes 

O No 

Not Applicable 


....for teachers to come on their own? 
O Yes 

O No 

Not Applicable 


8. To what extent do the library resources support your school’s literacy curriculums? 
1 Not atall 


To a small extent 


To a moderate extent 


To a large extent 
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9. Please use the table below to describe the nature of your collaboration with the school literacy team, grade level teams, individual teachers and 
your work with students during assigned library time. For each activity listed, 

a) in the first column indicate whether this is part of your responsibilities. 

b) Ifitis, indicate in the remaining columns which groups you collaborate with directly in support of that activity. 


a) b) IF YES: 
Is this part of | Check which groups you collaborate with in fulfilling this responsibility: 
Sy OUE a. Ps Grade Level cae Individual Working directly 
Responsibilities: responsibility’? Teams ae teachers with students 
Participate in collaborative decision-making about Ces CNo q 5 q q 
student literacy 
Identify and maintain high quality materials for Wes CINo q q q q 
recreational reading 
Maintain a middle-grade area in the library OYes ONo i) i) 0 0 
Refer struggling readers to summer programs that Ces CNo q q q q 
support literacy skills 
Provide students with information about extracurricular 
academic activities (e.g., spelling bee, writing Yes ONo Oo 0 0 o 
competitions, events) 
Develop summer reading activities for struggling readers Ces CINo q q q q 
who do not attend summer school 
Plan and conduct read-alouds OYes ONo 0 0 0 Oo 
Plan and conduct literature circles Yes ONo 0 0 0 0 
Plan and conduct book clubs OYes ONo 0 0 0 0 
Direct students to resources tied to the curriculum OYes ONo 0 0 0 0 
Identify resources for class projects Yes ONo Oo Oo Oo Oo 
Identify resources to supplement classroom lessons Yes ONo Oo =) Oo Oo 
Teach students how to navigate Internet resources OYes ONo 0 0 0 0 
Provide instruction on using print information resources OYes ONo 0 0 i) 0 
Provide instruction on using electronic information Ores ONo qo oq q q 
resources 
Teach students research skills OYes ONo 0 0 i) 0 
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9. Asa librarian, what do you need to better support the literacy needs of middle grade students? 


10. What do you need to improve collaboration with classroom teachers? 


About You 


11. What is the name of your school? [drop down list] 


Thank you for your time. 
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CPS Striving Readers 
Non-ELA Content Area Teachers LIST Section — Control Schools 


The following is a survey designed to gather your feedback on the essential components of your school’s 
literacy program. It will take you approximately 30 minutes to complete. Results will be reported in the 
aggregate only; we will not use your name or identify individual respondents. Your feedback is extremely 
valuable to the success of this program. If you have questions about this survey, please contact Rebecca 


Swann at rswann@metisassoc.com or 212-425-8833. 


This year 
Last school year 
Two or more years ago 


oooo 


Comprehensive Instruction 


When, if at all, did you start integrating literacy into your content area instruction? 


I do not integrate literacy instruction into my content area. (Skip to Question 6) 


2. How often do you use the following practices to help students increase reading comprehension? 


fee ress | 13 | 13 | 45 

Use of Instructional Practices we Never times a | times | times 
Familiar once a 
month | a week | a week 
month 

Explicit instruction in use of summarizing as a q q q q qo q 
comprehension strategy 
Explicit instruction in use of questioning as a q 4 q o 5 
comprehension strategy 
Explicit instruction in use of predicting as a q qo 4 q q 5 
comprehension strategy 
Explicit instruction in using text structure (the 
organizational arrangements used to present information) i) 0 Oo 0 im) Oo 
as a comprehension strategy 
Explicit instruction in use of visualization as a q o qo q o q 
comprehension strategy 
Explicit instruction in use of inferring as a 5 q 4 q 
comprehension strategy 
Explicit instruction in use of metacognition (students 
select appropriate comprehension strategies) as a 0 o 0 0 o o 
comprehension strategy 
Establishing the purpose for reading. i) im) Oo 0 im) 0 
Monitoring students’ comprehension through questioning. 0 0 Oo i) im) Oo 
Making connections to background knowledge. 0 im) Oo i) im) Oo 
Making connections between texts. i) im) Oo 0 im) 0 
Synthesizing information within text or across texts. 0 im) mt) 0 im) mt) 
Using differentiated instruction (using different 
instructional methods, modalities, materials, etc. for q 5 4 q 5 5 
different groups or individuals to address unique learning 
needs) 
Use of before, during, and after (BDA) reading strategies 
for comprehension instruction (A student constructed 
mental framework for reading begun before reading even 0 im) 0 0 im) 0 
begins, strengthened as students interact with the text 
during the reading, and reflected upon after reading.) 
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Not than i be as 
Use of Instructional Practices sas Never times a | times | times 
Familiar once a 
month | a week | a week 
month 
Use of the gradual release of responsibility model for 
reading comprehension instruction (Leading students from 
‘Modeled instruction’ to ‘Shared instruction’ to “Guided z e = 7 7 7 
practice’ and finally ‘Students’ independent practice’) 
Using partner reading to enhance comprehension q q 5 q q q 
instruction 
3. How often do you use the following practices to help students build their vocabulary knowledge? 
Less 1-3 1-3 4-5 
Use of Instructional Practices Nut Never SRL | ANNE MCS | SEES 
Familiar once a a a a 
month | month | week | week 
Explicit instruction in vocabulary o Oo 0 im) 0 0 
Modeling the use of word parts 0 
Review of vocabulary words im) 0 o o 0 im) 
Use of vocabulary notebooks o o o o o o 
Use of partner reading to enhance vocabulary 5 q q qo q q 
development. 
Use of before, during, and after (BDA) reading 5 q q qo q q 
strategies for vocabulary instruction 
Use of the gradual release of responsibility model for 7 5 q q 4 q 
vocabulary instruction 
4. How often do you use the following practices to help students develop fluency? 
Less 1-3 1-3 
Not than | tim tim oe 
Use of Instructional Practices a Never pee imes | times a 
Familiar once a a a k 
month | month | week ee 
Teacher read aloud i) 0 0 0 im) 0 
Teacher interactive read aloud 0 0 0 0 0 0 
Shared reading (students and teacher take turns in reading) 0 o 0 0 o 0 
Modeling reading for students 0 o o o o 0 
Explicit instruction in guided oral reading 0 o 0 0 0 0 
Focusing instruction on proper and meaningful phrasing 0 0 0 0 0 0 
Use of partner reading to enhance fluency instruction. 0 0 0 i) 0 i) 
Students listen to audio books, play-aways 0 o o o o o 
Use of the gradual release of responsibility model for 5 q q q qo q 


fluency instruction 
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5. How often do you use the following techniques to help students develop better reading strategies and 


skills? 
Techiiaues Not Never Less than 1-3 times | 1-3 times | 4-5 times a 
qu Familiar sb once a month | a month a week week 
Everybody Reads To (ERT) i) Oo i) 0 0 0 
Exclusion Brainstorming 0 0 0 0 0 0 
List-Group-Label i) Oo 0 0 0 0 
Predict-Locate-Add-Note (PLAN) 0 0 0 i) 0 0 
ReQuest o o o Oo o oO 
Interactive Notation System for Effective 
Reading and Thinking (INSERT) a 7 e Z o 7 
ABC Graffiti im) 0 Oo Oo 0 q 
Guided Reading and Summarizing 
Procedure (GRASP) 2 0 S 7 7 = 
6. Please indicate how often (if at all) you have met with each of the following literacy experts 
during the current school year. 
~~ He 1-3 1-3 4-5 
Meetings with Literacy Experts (Do not | Never times a | timesa | timesa 
have) oncea 
month week week 
month 
Lead Literacy Teacher/Literacy Coach Oo 0 0 im) 0 0 
Literacy Team Members 0 0 0 Oo 0 o 
Regular ELA teachers i) 0 0 0 Oo 
Other (specify): Oo 0 Oo 0 Oo Oo 
7. Please indicate how often (if at all) you discussed each of the following topics with any of the 
above literacy experts during the current school year. 
Less than ; : : 
Discussion Topics with a Literacy Expert Never meee. Oe ee ee 
a month a week a week 
month 
a. Differentiated instruction 0 0 0 0 0 
Student groupings Oo 0 0 0 0 
Use of text sets (content related books of 
different reading levels, genres and subject o Oo o o o 
themes) 
d. Use of technology resources: desktop 
computers, classroom computers/laptops, Oo 0 i) Oo Oo 
LCD projector, etc. 
e. Use of the partner-reading instructional q q 5 q q 
technique 
f. How to use specific instructional techniques q q 4 q 4 
for comprehension instruction 
g. How to use specific instructional techniques q q q q q 
for vocabulary instruction. 
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Less than | 1 3 times | 1-3 times | 4-5 times 
Discussion Topics with a Literacy Expert Never once a saninatin’ | aaedek meek 
month 
h. How to use specific instructional techniques q 0 oq 0 
for fluency instruction. 
i. Specific students’ reading progress. o o o 0 
j. Using student assessment data for q q q q 
instructional planning 
Grouping Structure 
8. How often do you use the following grouping structures in your classes? 
Less than : : c 
: 1-3 times a 1-3 times a | 4-5 times 
Grouping Structures Never once a 
month week a week 
month 
Whole class/Large group o 0 0 0 0 
Individual work o o Oo 0 i) 
Small groups o o o o o 
Partner Reading o o 0 0 0 
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School-wide Intervention Materials/Hide Qs 9-11 if response to QI = “I do not integrate literacy”] 
9. For the following school-wide intervention materials: 
= Indicate how frequently you currently use the materials to teach literacy through your subject area. 
= For those that you are using, rate how comfortable you are with using these materials to support student learning in language arts. 


Figen b) IF USED: 
a 
ey Rate your comfort level 
Materials N/A Not Less than | 1 to3 1 to3 4to5 1 5 
(Do Not | Currently | oncea times a times a times a Not at all 2 3 4 Very 
Have) Using month month week week | Comfortable Comfortable 
Text sets 0 0 0 0 0 0 0 0 0 0 0 
Media centers (classroom- 
based station with three 0 0 0 0 0 0 0 0 0 0 0 
computers and a printer) 
Listening centers 
(Classroom CD or cassette 
player, read-along audio 0 0 0 0 0 0 0 0 o o o 
books, playaways and 
headphones) 
Classroom library i) 0 0 im) 0 i) 0 0 0 0 0 
School library resources 0 0 0 0 0 0 0 o o o o 
Vocabulary notebooks i) i) i) 0 0 0 im) 0 0 0 0 
Textbooks 0 i) i) i) 0 i) 0 0 0 0 0 
Reading response notebooks 0 0 0 0 0 0 0 0 o o o 
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10. For each of the materials listed below, please indicate which grouping strategies are supported by your use of that 
material in your classroom when teaching literacy through your content area. (Check all that apply) 


Not Grouping Strategies 
Muterials Using | Whole Class’ | Small Group/ Individual 
Large Group Pairs Work 
Listening centers 0 oO im im) 
Media centers Oo og im) o 
Text sets and Text sets teacher guides Oo 0 Oo Oo 
Classroom library 0 i) o 0 
Vocabulary notebooks im) qo im) o 
Textbooks oO Oo o o 
Reading response notebooks 0 Oo im o 


Use of Text Sets and Text Sets Teacher Guides 


11. a) Do you use text sets (informational reading and listening materials at different levels and with varied text structures 
and organizational features that offer perspectives on a theme) to teach reading or literacy? 


Yes (SKIP to Question 12) 
No 


b) Ifyou are NOT yet using text sets, please indicate why you are not using them below (Check all that apply) 


They have not been made available to me 


The reading level is not appropriate for my students. 


The content is not relevant/interesting to my students 


I did not receive text sets for the topics that we are covering in my class 
Other (specify): 
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Professional Development in Literacy Instructional Practices 


12. For each of the following literacy teaching practices, indicate: 
a. Did you receive professional development on this topic during the current school year? 


If so: 


b. Please indicate whether you are using this practice as part of your content area instruction, and 
c. Rate your comfort implementing each teaching practice within your content area instruction. 


Literacy based teaching practices 


a) 


Received 
PD? 


IF YES: 


b) Using as part 


of content 
instruction? 


Z 
° 


m4 
2 


c) Rate your comfort level 


1 


Not at all 
Comfortable 


Z 
i) 
in 
& 
) 


as 


5 


Very 
Comfortable 


Building academic vocabulary 


Q 


mi) 


Using classroom libraries 


Creating literacy-rich classroom environments 


Differentiating instruction 


Direct vocabulary instruction 


Incorporating text sets in your instruction 


Increasing student motivation 


Supporting students’ self-directed learning 


sing before, during, and after reading strategies 


sing formal assessments to guide instruction 


sing informal assessments to guide instruction 


sing classroom computers 


sing literacy-based software 


U 
U 
U 
U 
U 
U 


sing partner-reading 


OQ)O;)/O/;,O;/O/;0;/O0/0/0;0/0/0/;0/;0 


O)O;/OV)O;O/;O;/O/O/0;0/;/0/0;0);0 


OQ)O;QO;,Q0;/0;O0/;/Q0/0/);0/;/0/0;0);0/0 
OQ)O;/QO;,0O/0;/O0/;/0/0/;/0/;/0/0;0);0/0 


O)O;/O)O;O/;0O;/O/0/0;0/;/0/0;0);0 
OQ)O;O,/O;/0;O0/0/0;0/;/0;0;0)/0 


O)O;)/O/;,O;/O/;0;/O/0/0;0/;/0/;/0);0/;0 


OQ)O;QO;/Q0;/0;0/;/0/0/;/0/;/0/0;0);0/0 


QOQ)O;);O,O;/0;O0/0/0;0/;/0;0;0)/0 
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Respondent Information 


13. At which grade level(s) are you teaching this year (2008-09)? (Check all that apply): 


0K010203040506070809010011 012 


14. What is the name of your school? [drop down list] 


15. How many years have you been teaching? [INSERT TEXTBOX] 
16. How many years have you been teaching at this school? [INSERT TEXTBOX] 


17. How many years have you been teaching in your subject area? [INSERT TEXTBOX] 


Teachers will be reimbursed by CPS-Striving Readers for their time to complete this survey. In order to be 
reimbursed we need you to identify yourself so that we can verify that you completed the survey. If you would like 
to be reimbursed, please provide your name and email address below, and be sure that you identified your school in 
the previous item. Your survey responses will still remain strictly confidential and will never be reported in any 
form that would allow anyone to connect your responses with your name. Providing this information is optional. 


Your Name: Email: 


Thank you for completing this survey! 
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SPRING 2009 DISTRICT-WIDE PRINCIPAL INTERVIEW 


TREATMENT SCHOOLS 
Interviewee Name: Date: 
Interviewee Title: 
School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. For this study, we are surveying and interviewing staff both 
from schools that are using the Striving Readers curriculum, and those from a comparison sample of 
schools that are not using Striving Readers. We are interested in learning about the literacy interventions 
for grades 6 through 8, so please respond to all of the questions as they relate to those grades only. Since 
we are requesting a lot of information and we have a limited period of time, most of the questions will be 
short-answer; however, please feel free to comment on any questions that you would like to. You will 
also have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. Your responses will be kept confidential. We don't identify 
individual respondents or their schools. I would like to tape this interview to be sure I have recorded it 
accurately. Is this all right? 


1. Has your role in the Striving Readers Initiative changed since last year? 


[| DYes 
L] No (SKIP TO Q2) 


a. Ifso, how? 
Literacy Leadership Teams 
2. Does your school have a Literacy Team? 


OYes 
ONo (IF NO, SKIP TO Q3) 


a. Which of your staff are members of the Literacy Team? 


1] Principal (| Grade level teacher(s) 1] Literacy Intervention Teacher 
(| Librarian(s) (| Lead Literacy Teacher [|] ELL/ESL Teacher(s) 
(| Special education teacher(s) (| Other: 


b. How often does the Literacy Team meet? 


OHas not met OLess than once per month Once per month 
OBiweekly OWeekly Several times a week or more 
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c. Overall, rate the quality of the literacy team’s performance in the following areas. 


Poor Fair Good Excellent | Not Sure 
Addressing the needs of all students qo q q q qo 
Addressing the needs of struggling q q q q q 
readers. 
Addressing the needs of grade-level q q q q q 
teams. 
Addressing the needs of individual q q q 4 4 
teachers 
Addressing school wide needs (grades 6- 
8) included in SIPAAA a - a er = 
Using assessment data and or student q q q q q 
work to drive instruction 
Supporting vertical and horizontal teacher q q q q q 
collaboration 
Improving literacy instruction at your 4 5 7 4 4 
school. 


Comments: 


3. Does your school have grade level teams? 


OYes 
ONo (IF NO, Skip to Q4) 


a. Which of your staff are members of the grade level teams? 


(iPrincipal 


LIELA teacher(s) 


(\Librarian(s) () Other: 


_|Content area teachers 


(iLead Literacy Teacher(] ELL/ESL Teacher(s) 


b. How often do the grade level teams meet? 


OHas not met OLess than once per month 


OBiweekly OWeekly 


Literacy Intervention Teacher 


Special education teacher(s) 


Once per month 


OSeveral times a week or more 
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c. Overall, rate the quality of the grade level team’s performance in the following areas. 


Poor Fair Good iets Not Sure 
Addressing the needs of all students im) qo q qo qo 
Addressing the needs of struggling readers. og 0 gq q q 
Using assessment data to plan instruction og og qo q q 
Using assessment data to establish vertical 
and horizontal literacy goals by grade level a a z a 
Improving literacy instruction at your q qq q 4 q 


school. 


Comments: 
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Use of Assessment Data 


We would like to learn more about the use of assessment data and how that impacts instruction. 


4. In what ways, if any, is your school using student assessment data beyond mandated reporting to the district and state? For each of they uses 
of assessment data listed below, please indicate: 
a. the extent to which student assessment data in your school are used for each purpose.; and, 
b. for those uses to which assessment data are being applied, which individuals or groups are using the data in this way. 


ay To whatextent? b) IF USED for this purpose: 


By whom? 
Panctnal ELA Literacy 
Is student assessment data used for this ete Teachers Intervention 
urpose? Toa Tee Toa Devent : (beyond Teacher 
PUEP ; Not at moder involvement | Literacy | - 
small large |. involvement (beyond Other: 
all ate in the Team : ‘ . 
extent extent . in the involvement in 
extent Literacy . : 
Team) Literacy the Literacy 
Team) Team) 
Screening students’ ability levels for placement q 5 q 5 q q q 4 q 


in intervention programs 


Diagnosing students’ strengths and support 
needs for placement in specific courses or o 0 im) 0 qo q 0 im) qo 
instructional groups 


Identifying trends in fluency and comprehension 
abilities across groups of students 


Identifying trends in vocabulary knowledge 
across groups of students 


Monitoring overall student progress for the 
purpose of assessing success of instructional im) Oo q Oo qo qo 0 q qo 
programs and methods 


Differentiating instruction qo o q o qo qo 0 q qo 
Planning on-site professional development q o qo o qo oq im qo Oo 
Other 0 0 i) 0 0 0 im) Oo o 
Other i) im) i) i) 0 0 im) i) i) 
Comments: 
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AMPAfter-School Literacy Program 


5. Overall, how appropriate would you say that the AMP after-school program is to the reading levels and 
needs of the students who are currently participating? 
Oo Notat all appropriate 


oo°o 


Somewhat appropriate 
Appropriate 
Very appropriate (Skip to Q6) 


6. Overall, how effective has the AMP after-school component been in improving the literacy 
abilities of struggling readers? 


Not at all | Minimally | Somewhat Effecti Very Don’t 
effective effective effective Wee effective know 
| oO | | | oO 


Comments: 


Integration of Literacy Instruction in Content Areas 


7. We would like to know more about your school’s efforts to integrate literacy into the content 


areas. 


To what extent do non-literacy teachers integrate 


To what extent? 


literacy into the content areas? Not at all | 1°2Small | Toamoderate | Toa large 
extent extent extent 
Math 0 im) o o 
Social Studies 0 0 o o 
Science 0 im) o o 
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8. Through the Striving Readers program, all participating schools received a series of text sets(i.e., sets 
of non-fiction reading materials of different structures and levels, centered around specific content 
area themes, designed to improve student literacy in other subject area classes) with accompanying 


teacher guides. 


Are these text sets being used in the content area classrooms in each subject? 
Social Studies 


OWYes 
OLINo 
OODon’t Know 


If YES: How are they used? 
If NO: Why not? [Probe: distribution problems, teachers’ access, alignment with curriculum] 


Science 
OOYes 
OUNo 
OUDon’t Know 


If YES: How are they used? 
If NO: Why not? [Probe: distribution problems, teachers’ access, alignment with curriculum] 


Mathematics 
OWYes 
OLINo 
OODon’t Know 


If YES: How are they used? 
If NO: Why not? [Probe: distribution problems, teachers’ access, alignment with curriculum] 
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9. Have staff at your school other than ELA teachers been attending professional development for the 
Striving Readers project? (Includes: Bilingual, SPED, Math, Science, Social Studies teachers) 


OWYes 
OLINo 


If YES: 


Who has received professional development? Please specify staff positions: 


In what topics did they receive training? (check all that apply) 


Building academic vocabulary Supporting students’ self-directed learning 
Using classroom libraries OWsing before, during, and after reading strategies 
Creating literacy-rich classroom environments Using formal assessments to guide instruction 
ODifferentiating instruction OWsing informal assessments to guide instruction 
ODirect vocabulary instruction Using classroom computers 
OIncorporating text sets in your instruction Using literacy-based software 
OlIncreasing student motivation OWsing partner-reading 
Other: 
Other: 

If NO: 


Why haven’t they been participating? 


Who provided this training? (Check all that apply) 
OIn-house staff. Please specify: 

ODistrict experts. Please specify: 
Outside consultants (e.g., literacy experts, universities, organizations). Please specify: 


Has any of your staff taken any courses towards their Masters or English language arts 
endorsement in NationalLouisUniversity as part of Striving Readers? 

Oj Yes (Who, when) 

O1No 


A-284 


Technology 


13. For each of the following technology resources, please indicate: 
a. if your teachers are using this resource to teach literacy 
b. for those resources your teachers are not using, please indicate why (Check all that apply) 


Resources Using? If NO, why not? (Check all that apply) 
software Students do not teachersdo not | I do not feel that they 
Vee Ne sas applications have sufficient feel comfortable offer sufficient added Other 
working have not been access to the using the benefit compared to (please specify): 
properly installed resources technology traditional media 
Media Centers 
(classroom-based 
station with 00 0 Oo 0 o 0 
computers and a 
printer) 
Listening Centers 
(Classroom CD or 
cassette player, read- 00 i) 0 Oo o Oo 
along audio books and 
headphones) 
Handheld 
Computers oo oO Oo o o o 
(Palm Pilots) 


A-285 


14. For each of the following technology resources that your teachers are using to teach literacy, 
please indicate to what extent that technology resource is integrated into the literacy 


curriculum. 
Extent integrated 
Resources NA Not at all Somewhat Thoroughly 
integrated integrated integrated 
Media Centers 
(classroom-based station with q q q q 
desktop computers and a 
printer) 
Listening Centers 
(Classroom CD or cassette player, 
read-along audio books and = EY a ie 
headphones) 
pangneld Computers q q q q 
(Palm Pilots) 


20. Overall, how well is technology integrated into the literacy curriculum? Would you say it is: 


ONot at all integrated Somewhat integrated OThoroughly integrated 


Comments: 


21. What impact would you say technology has had on the reading achievement of struggling 
readers in your school? Would you say it had: 


ONo impact OSomeimpact OModerate impact OLarge impact 


Please explain your rating: 


Professional Development 


22. Have you participated in Striving Readers professional development related to literacy this 
school year or last summer? 

110 Yes 

(| O No (SKIP TO Q17) 
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23. For each of the following Striving Readers professional development sessions conducted during the 2008- 
2009 school year, please indicate: 


=" Whether you participated, and 


= Ifso, how useful the session(s) was (were) in helping you support student learning in 
language arts 


ahs poe If YES, how useful was the session? 
: participate? 
Professional Development Sessions 
Not Somewhat Moderately 
No Yes Useful Useful Useful Extremely Useful 
Monthly Principals Meetings (Leaders q q q q q 4 
Seminars) 
2008 Summer institute 0 0 0 0 0 0 
School-year follow-up institutes i) 0 i) 0 0 0 
Saturday seminars 0 0 0 0 0 0 
On site training during literacy team q q q q q q 
meetings 
School-based Striving Readers q q qo q q q 
professional development 


Comments: 


Perceptions of the Literacy Curriculum 


24. Overall, what are the strengths of your school’s literacy curriculum? 


25. Overall, what are the challenges to your school’s literacy curriculum? 


26. What does your school need to better support literacy instruction? 


27. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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SPRING 2009 DISTRICT-WIDE PRINCIPAL INTERVIEW 


CONTROL SCHOOLS 
Interviewee Name: Date: 
Interviewee Title: 
School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. For this study, we are surveying and interviewing staff both 
from schools that are using the Striving Readers curriculum, and those from a comparison sample of 
schools that are not using Striving Readers. We are interested in learning about the literacy interventions 
for grades 6 through 8, so please respond to all of the questions as they relate to those grades only. Since 
we are requesting a lot of information and we have a limited period of time, most of the questions will be 
short-answer; however, please feel free to comment on any questions that you would like to. You will 
also have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. Your responses will be kept confidential. We don't identify 
individual respondents or their schools. I would like to tape this interview to be sure I have recorded it 
accurately. Is this all right? 


1. Has your role in supporting the literacy instruction that takes place at your school changed 
since last year? 


OYes 


ONo (SKIP TO Q2) 
a. Ifso, how? 


Literacy Leadership Teams 
2. Does your school have a Literacy Team? 


OYes 
ONo (IF NO, SKIP TO Q3) 


a. Which of your staff are members of the Literacy Team? 


(| Are youamember? (Grade level teacher(s) 


(| Lead Literacy Teacher L]| ELL/ESL Teacher(s) 


(| Reading Specialist 


(| Librarian(s) 


(| Special education teacher(s) (| Other: 


b. How often does the Literacy Team meet? 


OHas not met OLess than once per month Once per month 
OBiweekly OWeekly Several times a week or more 
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c. Overall, rate the quality of the literacy team’s performance in the following areas. 


Poor Fair Good Excellent | Not Sure 
Addressing the needs of all students og q q q q 
Addressing the needs of struggling readers a) og gq q q 
Addressing the needs of grade-level teams q q q q q 
Addressing the needs of individual q q q q 4 
teachers 
Addressing school wide needs (grades 6- 
8) included in SIPAAA = a a zs 2 
Using assessment data and or student q q q q q 
work to drive instruction 
Supporting vertical and horizontal teacher q q q q q 
collaboration 
Improving literacy instruction at your 4 5 q 5 q 
school 


Comments: 


3. Does your school have grade level teams? 


Yes 
LIGNo (IF NO, Skip to Q4) 


a. Which of your staff are members of the grade level teams? 


(|Are you a member? 


LIELA teacher(s) 


(\Librarian(s) (jOther: 


b. How often do the grade level teams meet? 


Content area teachers 


Has not met (10) Less than once per month 


(| OBiweekly (10 Weekly 


(jReading Specialist 


(JLead Literacy Teacher(] ELL/ESL Teacher(s) Special education teacher(s) 


(| OOnce per month 


OSeveral times a week or more 
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c. Overall, rate the quality of the grade level team’s performance in the following areas. 


Poor Fair Good | Excellent | Not Sure 
Addressing the needs of all students a) qo q qo qo 
Addressing the needs of struggling readers a) 0 im) o q 
Using assessment data to plan instruction og o a) qo q 
Using assessment data to establish vertical 
and horizontal literacy goals by grade level eo 2 a z a 
Improving literacy instruction at your school og o og qo q 


Comments: 


No comments. 
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Use of Assessment Data 


We would like to learn more about the use of assessment data and how that impacts instruction. 


4. In what ways, if any, is your school using student assessment data beyond mandated reporting to the district and state? For each of the uses 
of assessment data listed below, please indicate: 


a. the extent to which student assessment data in your school are used for each purpose; and, 
b. for those uses to which assessment data are being applied, which individuals or groups are using the data in this way. 


a) To what extent? b) IF USED for this purpose: By whom? 
Principal ELA Reading 
Is student assessment data used for this Toa (beyond Teachers Specialist 
Not at de moder tide involvemen | Literac (beyond (beyond 
purpose? small : large mec stan | Other: 
all ate tin Team involvement | involvement in 
extent extent : Se : 
extent Literacy in Literacy Literacy 
Team) Team) Team) 
Screening students’ ability levels for placement 
& y Pee o oO oO oO Oo Oo oO o oO 


in intervention programs 


Diagnosing students’ strengths and support 
needs for placement in specific courses or q im) q 0 im) im) q o q 
instructional groups 


Identifying trends in fluency and comprehension 


abilities across groups of students 0 0 0 0 0 0 im) 0 o 


Identifying trends in vocabulary knowledge 
across groups of students 


Monitoring overall student progress for the 
purpose of assessing success of instructional q a) q 0 im) im) q o q 
programs and methods 


Differentiating instruction qo Oo qo o go im) q Oo q 
Planning on-site professional development o o q Oo o o qo o qo 
Other 0 0 0 im) 0 im) 0 o 0 
Other 0 0 0 0 Oo 0 0 o 0 
Comments: 
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After-School Literacy Program 
5. a) Does your school have an onsite, after-school intervention program? 


OYes, 
ONo(Skip to Q8) 


b) Who is the afterschool program targeting? 
Struggling Readers Only 

Oj Students at or Above Grade Level (Skip to Q8) 
OAI Students, including struggling readers 


c) What criteria are used to determine students’ eligibility for the after-school literacy program? 
[Probe: test scores (specific cut-offs?); other literacy assessments (specify type and cut-off criteria); 
teacher referrals; etc.] 


Criteria: 


6. a) Overall, how appropriate would you say that the after-school program is to the reading levels and 
needs of the students who are currently participating? 
Oo Notat all appropriate 
oO Somewhat appropriate 
O Appropriate 
oO Very appropriate 


7. Overall, how effective has the after-school literacy program been in improving the literacy 


abilities of struggling readers? 


Not at all | Minimally | Somewhat Effecti Very Don’t 
effective effective effective Wee effective know 
| oO | | | oO 


Comments: 


Integration of Literacy Instruction in Content Areas 


8. We would like to know more about your school’s efforts to integrate literacy into the content 


areas. 

To what extent do non-literacy teachers integrate Po whavexteahe 

literacy into the content areas? Not at all | 1°2Small | Toamoderate | Toa large 
extent extent extent 

Math Oo Oo Oo oq 

Social Studies Oo 0 Oo 0 

Science oO 0 =) 0 
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9. Do you use text sets(i.e., sets of non-fiction reading materials of different structures and levels, 
centered around specific content area themes, designed to improve student literacy in other subject 
area classes) and their accompanying teacher guides? 


Are these text sets being used in the content area classrooms in each subject? 
Social Studies 
OYes 


OINo 
OODon’t Know 


If YES: How are they used? 
If NO: Why not? [Probe: distribution problems, teachers’ access, alignment with curriculum] 


Science 
OWYes 
OUNo 
OUDon’t Know 


If YES: How are they used? 
If NO: Why not? [Probe: distribution problems, teachers’ access, alignment with curriculum] 


Mathematics 


OWYes 
OWNo 
OUDon’t Know 


If YES: How are they used? 
If NO: Why not? [Probe: distribution problems, teachers’ access, alignment with curriculum] 


A-293 


Technology 


10. For each of the following technology resources, please indicate: 
a. if your teachers are using this resource to teach literacy 
b. for those resources your teachers are not using, please indicate why (Check all that apply) 


Using? If NO, why not? (Check all that apply) 
Not software Students do not teachersdo not | I do not feel that they 
We do not : applications have sufficient | feel comfortable | offer sufficient added Other 
Yes No working : ; 
have RE have not been access to the using the benefit compared to (please specify): 
Resources BrODery installed resources technology traditional media 
Media Centers 
(classroom-based 
station with 00 0 0 Oo 0 0 0 
computers and a 
printer) 
Listening Centers 
(Classroom CD or 
cassette player, read- oo 0 0 0 0 o 0 
along audio books and 
headphones) 
Laptop computers 
005 0 0 0 0 0 im) 
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11. For each of the following technology resources that your teachers are using to teach literacy, 
please indicate to what extent that technology resource is integrated into the literacy 


curriculum. 
Extent integrated 
Resources NA Not at all Somewhat Thoroughly 
integrated integrated integrated 
Media Centers 
(classroom-based station with q q q q 
desktop computers and a 
printer) 
Listening Centers 
(Classroom CD or cassette player, 
read-along audio books and z 7 2 o 
headphones) 
Laptop computers o o Oo im) 
Other: 5 4 4 
Smart Boards 


12. Overall, how well is technology integrated into the literacy curriculum? Would you say it is: 


OUNot at all integrated (J0Somewhat integrated (| OThoroughly integrated 


Comments: 


13. What impact would you say technology has had on the reading achievement of struggling 
readers in your school? Would you say it had: 


ONo impact OSomeimpact OModerate impact OLarge impact 


Please explain your rating: 


Professional Development 


14. Has any literacy-related professional development been made available for staff at your school 
during this school year or last summer? 


(JO Yes 
CIGNo (Skip to Question 17) 
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If YES: 


On what topics was training provided? (check all that apply) 


OBuilding academic vocabulary OU Supporting students’ self-directed learning 

OOUsing classroom libraries OUUsing before, during, and after reading 
strategies 

Creating literacy-rich classroom environments OU Using formal assessments to guide instruction 

ODifferentiating instruction OOUsing informal assessments to guide instruction 

OUDirect vocabulary instruction OUsing classroom computers 

OU Incorporating text sets in your instruction OU Using literacy-based software 

OU Increasing student motivation OU Using partner-reading 

OU Other: 

OW Other: 

OW Other: 


Who provided this training? (Check all that apply) 


OIn-house staff. Please specify: 
ODistrict experts. Please specify: 


Outside consultants (e.g., literacy experts, universities, organizations). Please specify: 


15. [IF NOT MENTIONED IN RESPONSE TO ABOVE]: Have you participated in professional 
development related to literacy this school year or last summer? 


(iTYes 
LI No (SKIP TO Q17) 


If yes, please describe: 


[prompt: topics, who conducted training, who participated] 


16. [IF NOT MENTIONED IN RESPONSE TO ABOVE]: Have any non-literacy staff at your school 
been attending professional development related to literacy this school year or last summer? 
(Includes: Bilingual, SPED, Math, Science, Social Studies teachers) 


OWYes 


Who has received professional development? Please specify staff positions: 


What topics were covered? 


Same as above 


OINo 


Why haven’t they been participating? 
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Perceptions of the Literacy Curriculum 


17. Overall, what are the strengths of your school’s literacy curriculum? 
18. Overall, what are the challenges to your school’s literacy curriculum? 
19. What does your school need to better support literacy instruction? 


20. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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Project Director Interview Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and am one of the members of the Striving Readers evaluation team 


at Metis. We are interviewing members of the SR district leadership team to learn more about the 
program; this interview will take about 75 minutes. I would like to tape this interview to be sure I have 
recorded it accurately. Is this all right? 


1. What is your role as the Project Director of the Striving Readers program? 
a. Have your responsibilities changed during the past three years? If so, how? 
b. What is your role in monitoring program implementation? 
c. What other responsibilities do you have (separate from Striving Readers)? Has the extent or 
type of these responsibilities changed over time? How do these responsibilities impact your 
ability to fulfill your role in SR? 


2. Please describe your work with the following key Striving Readers staff. Has your work with 
these staff changed over time? If so, how? 
a. District coordinators. 


7 How do you communicate with them? 
" How often do you meet with each coordinator individually? As a group? 
7 How do you determine individual responsibilities on the project? 
7 Describe how you ensure accountability for the coordinators accountable? 
7 What types of support do you provide to district coordinators? 
b. Senior Literacy Consultant 
7 How do you communicate with her? 
7 How often do you meet with her? 
7 What is the nature of the collaboration? 
c. District Reading and Language Manager 
7 How do you communicate with him or her? 
7 How often do you meet with her? 
7 What is the nature of the collaboration? 
d. Individual schools. 
7 How often do you visit the schools? 
7 With whom do you meet (e.g., principals, LITs, teachers, students, librarian, 
tech coordinator, other)? 
7 What is the nature of the collaboration? 
7 Do you conduct observations of instruction? 
7 Have you had sufficient opportunity to follow up on these visits to the extent 


you would like? 
3. How do you anticipate that the recent district-level changes, such as the restructuring of the 


district or new or newly expanded initiatives, will affect implementation of SR and the program 
as a whole? (Probes: district staffing changes; restructuring from Office of Literacy to Office of 


A-298 


Reading and Language Arts; citywide expansion of SCRMA and associated changes in literacy 
materials) 


4. Describe the support you receive from the district for this program. 
a. Has the support changed over time? If so, how? 


5. How have school-based changes and/or restructuring affected program implementation? (Probe 
for: changes in school administrations, changes from self-contained to departmentalized structure.) 
a. How are district SR leadership adapting to these changes? 


6. To what extent has the district SR leadership been able to convey expectations of accountability 
for program implementation? How has this been accomplished? 
e Have recent district-level changes affected this process? 
e What impediments, if any, have been encountered to establishing accountability? 
e To what extent do you believe that school administrators are communicating these expectations 
and holding their staff accountable? 


7. We would like to learn more about the District-Level Team’s use of data to inform project 
management. In what ways, if any, does the district-level team use the following types of data to 
inform project management? (Probe for: rubrics, surveys, implementation/evaluation data, 
professional development, assessment, and other, data.) 

a. Has the team used these data to make adjustments to address implementation challenges? 
Please provide examples. 


8. Now I would like to ask some questions about the professional development plan for Year 3 (SY 
2008-09). 

a. Did you personally facilitate professional development activities during year 3? If so, how 
did this fit into the overall PD plan for SR? 

b. Considering the overall PD plan for Year 3, in what ways, if any, was professional 
development differentiatedfor different schools and different staff? 

c. Have there been any changes over time? If so, why? 

d. How many new teachers were in Striving Readers schools in Year 3? Were they offered 
specific professional development opportunities (or will they be in Yr 4)? 

e. What were the successes of the professional development activities of Year 3? 

f. What challenges did you encounter with the implementation of professional development 
activities in Year 3? How were these challenges addressed? 


9. In what ways has the Chicago Striving Readers program used each of the following types of 
technology as a tool to help improve differentiated literacy instruction for readers at all levels? 
What have been the successes and challenges of these efforts? To what extent have schools succeeded 
in using these technologies to support differentiated instruction? 

e Classroom media centers” 
e Listening centers® 
e Handheld computers (Palm Pilots) 


°3 computers and a printer 
°students access models of fluency, record and listen to themselves reading,and listen to audio books 
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10. 


11. 


12. 


13. 


To what extent has program implementation expanded into upper grades and into other subject 
areas in the past year? Please describe factors facilitating or hindering this process.(Probes: role of 
SR text sets, PD for grade 7 and 8 teachers and for non-literacy staff, variations among schools) 


Describe the major successes of the Striving Readers program in Year 3 (SY 2008-2009). When 
possible, please differentiate between the blended model, the targeted intervention and the 
intensive intervention. 


What are some of the challenges that have been encountered in Year 3 (SY 2008-2009)? When 
possible, please differentiate between the blended model, the targeted intervention, and the 
intensive intervention. 

a. How have these challenges been addressed or how will they be addressed in Year 4? 


Is there anything else you would like to add regarding the program or literacy activities for 
struggling readers in the district? 
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Literacy Consultant Interview Protocol 


Interviewee Name: — Donna Ogle Interviewee Title: 

Date: Start Time: End Time: 
Interviewer: 

Introduction: I'm ; Lam one of the members of the Striving Readers evaluation team at 


Metis. We are interviewing members of the SR district leadership team to learn more about the program; 
this interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. Is this all right? 


1. What is you role as Senior Literacy Advisor of the Striving Readers program? 
a. Have your responsibilities changedduring the past three years? If so, how? 
b. Describe your role in project-level components of Striving Readers (Probe for: 
participation in summer institutes; 
participation in Saturday seminars; 
coordination of coursework with National-LouisUniversity; 
establishing goals and identifying and addressing challenges to meeting them). 
Have these roles changed during the past three years? If so, how? 


2. Please describe your work with the following key Striving Readers staff. 
a. Project director. 


7 How do you communicate with her? 
7 How often do you meet with her? 
7 What is the nature of the collaboration? 
7 If at all, how does your work with the Project Director strengthen project 
leadership? 
b. District coordinators. 
“ How do you communicate with them? 
7 How often do you meet with them? 
7 What is the nature of the collaboration? 
c. Individual schools. 
7 How often, if at all, do you visit the schools? 
7 How often, if at all, do you work directly with the principals of the schools? 
7 With whom else do you meet (e.g., teachers, students, librarian, tech 
coordinator, other)? 
7 Do you conduct observations of instruction? 
a What is the nature of the collaboration? 


3. We would like to learn more about the District-Level Team’s use of data to inform project 
management. In what ways, if any, does the district-level team use the following types of data to 
inform project management? (Probe for: rubrics, surveys, implementation/evaluation data, 
professional development, assessment, and other, data.) 

a. Has the team used these data to make adjustments to address implementation challenges? 
Please provide examples. 
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How do you anticipate that the recent district-level changes, such as the restructuring of the 
district or newly expanded initiatives, affected your relationship with the district staff and/or 
your work on the Striving Readers program?How have they affected program 
implementation?(Probes: district staffing changes; restructuring from Office of Literacy to Office of 
Reading and Language Arts; citywide expansion of SCRMA; shift from self-contained to 
departmental structure in the six through eighth grades.) 


In its first two years, the SR program made substantial progress in increasing the amount of 
small group instruction that takes place in literacy classes. What progress do you feel the 
project has now made in moving beyond small group instruction into authentic use of 
differentiated instruction? What successes and challenges have been encountered in these efforts 
during Year 3? 


Describe the major successes of the Striving Readers program in Year 3 (SY 2008-2009). When 
possible, please differentiate between the blended model, the targeted intervention and the 
intensive intervention. 
a. Do you feel that the program is reaching maturity? How do you think “maturity” should be 
defined? 


What are some of the challenges that have been encountered in Year 3 (SY 2008-2009)? 
a. How were these challenges addressed last year and how will they be addressed in year 4? 


What do the schools need to better support literacy instruction? 


Is there anything else you would like to add regarding the program or literacy activities for 
struggling readers in the district? 
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School Coordinator Interview Protocol 


Interviewee Name: Date: 
Interviewer: Start Time: End Time: 
Introduction: I'm and am one of the interviewers with the Chicago Public Schools 


Striving Readers evaluation team at Metis. We are interviewing members of the SR district leadership 
team to learn more about the program; this interview will take about one hour. I would like to tape this 
interview to be sure I have recorded it accurately. Is this all right? 


1. What is you role as a District Coordinator of the Striving Readers Program? 
a. Have your responsibilities changed over time? If so, how? 
b. What training have you received to help you execute these responsibilities? 


2. Please describe how you work with the individual schools. 
a. How often do you visit each school? 
b. With whom do you primarily work? What is the nature of your collaboration? Has your work 
with the following stakeholders changed over time? If so, how? 


7 School administration 

7 Teachers 

" Literacy Intervention Teacher 
" Students 

7 Librarian 

7 School technology coordinator 
" Other? 


3. To what extent (and in what ways) do you communicate SR expectations to school 
administrators and teachers and ensure accountability for the implementation of Striving 
Readers? (Probes: focus is as much on how they define the expectations as on how they communicate 
them. How do they monitor program implementation at the school level? The classroom level?) 

a. 


4. Describe the support you receive from the district for this program. 
a. Has the support changed over time? If so, how? 
b. Do you have other non-SR competing priorities? If so, how do they affect your SR work? 


5. What supports do you provide to the school? (Probes: support of data collection and analysis 
processes; collaboration with LITs; role in school literacy team meetings; what types of on-site PD 
do you offer and to whom?) 


6. How do you anticipate that recent district-level changes, such as the restructuring of the district 
or new or newly expanded initiatives, might affect the SR program? (Probes: district staffing 
changes; restructuring from Office of Literacy to Office of Reading and Language Arts; citywide 
expansion of SCRMAand associated changes in literacy materials) 


7. How have school-based changes and/or restructuring affected program implementation at the 


school level? On the program as a whole? (Probe: changes in school administrations, changes 
from self-contained to departmentalized structure) 
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8. We would like to learn more about the leadership team responsible for implementation of 
Striving Readers at the district level. 
a. Who comprises this district-level team? 
b. How often do you meet with other District Coordinators? With the Project Director? With the 
Literacy consultant [Donna Ogle]? 

c. What is the nature of this collaboration? 
7 What topics do you discuss? 
® How does the district-level team respond to challenges to implementation of 

the Striving Readers program? Please provide some examples. 


9. What types of data, if any, do you and/or the district team use to inform your work? (Probe for: 
implementation, professional development, student assessment, evaluation data) 
a. Please explain how and for what purposes you use each type of data. 


Next, we would like to know more about this year’s implementation of SR in grades 7 and 8, in non- 
ELA classrooms, and in schools of different sizes. 


10. To what extent is SR being implemented in the seventh and eighth grades? 
a. Does this vary by school and/or teacher? Please describe factors facilitating and/or hindering 
this process. 
b. Has the expansion of the SCRMA initiative affected the extent of implementation? 


11. To what extent are non-literacy teachers integrating literacy instruction into their content areas 
as part of the Striving Readers program? 
a. Does this vary by school and/or teacher? Please describe factors facilitating and/or hindering 
this process. 
b. Has the expansion of the SCRMA initiative affected the extent of integration? 


12. Have you observed any differences in program implementation between smaller and larger 
schools in each of the following areas: 

Work of the LIT (e.g., grade-levels with which they work, intensity, collaboration with 

teachers) 

Availability of staffing such as literacy coaches, librarian or technology coordinator 

Structures such as grade-level teams and/or literacy team 

Intersection between SCRMA initiative and Striving Readers 

Other 


ones 


13. What have been the major successes and challenges in utilizing technology to improve 
differentiated literacy instruction? (Probe: availability of hardware, software, teacher comfort 
level; use to support differentiated instruction, not just small group instruction.) 

a. Handheld Computers (Palm Pilots) 
b. Media Centers 
c. Listening Centers 


14. Describe the major successes of the Striving Readers program in Year 3. When possible, please 
differentiate between the blended model, the targeted intervention and the intensive 
intervention. 

a. What district-level, school-level and classroom-level factors facilitate the implementation and 
success of the program at each of these levels? 
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15. 


16. 


17. 


What are some of the challenges that have been encountered in Year 3? When possible, please 
differentiate between the blended model, the targeted intervention and the intensive 
intervention. 

a. How have these challenges been addressed or how will they be addressed in year 4? 


What do the schools need to better support literacy instruction? 


Is there anything else you would like to add regarding the program or literacy activities for 
struggling readers in the district? 
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Technology Coordinator Interview Protocol 


Interviewee Name: Title: 
Interviewer: Date: Start Time: End Time: 
Introduction: I'm ;] am one of the members of the CPS Striving Readers evaluation 


team at Metis. We are interviewing members of the SR district leadership team to learn more about the 
program; this interview will take about one hour. I would like to tape this interview to be sure I have 
recorded it accurately. Is this all right? 


1. What is your role as [Technology Coordinator/Technology Consultant] of the Striving Readers 
program? 
a. Have your responsibilities changed over time? If so, how? 
b. How does your role relate to that of the other Technology Co-Coordinator [Rob 
Residori/Lamarr Wilson]? Please describe how you work together. 


2. Please describe the technology support provided to individual schools. 
a. How often do you visit each school? 
b. Who do you primarily work with? What is the nature of your collaboration? 


7 Principals 

7 Teachers 

7 Students 

7 Librarian 

" School technology coordinator 
7 Other? 


c. Apart from your own services, do the schools receive any other technology-focused support? 
d. Are different levels of support provided for schools that previously had no technology? 


3. Do you differentiate your school support and or professional development services? 
a. If yes, at what level (e.g., individual, staff type, grade, cluster, focus area)? 
b. How do you determine school or teacher needs? 
c. Has the shift from a self-contained to a departmentalized structure in the six through eighth 
grades impacted how you provide technology support? 
d. How many new teachers were in Striving Readers schools in Year 3? Were they offered 
specific professional development opportunities (or will they be in Yr 4)? 


4. Describe the support you receive from the district for this program. 
a. Has the support changed since over time? If so, how? 


5. In what ways is the Chicago Striving Readers program using Handheld computers (Palm 
Pilots) as a tool to help improve differentiated literacy instruction for the following groups of 
students? Please respond to each of the questions below and indicate how the status differs for 
each of the following groups: 

e Whole class/blended instruction model (all students) 
e In-class targeted intervention for Tier 2 and 3 students 
e After-school Intensive intervention for Tier 3 students 
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For each of the above groups (where applicable)... 
[Probe for each: What have been the successes and challenges of these efforts? How have these 
successes and challenges changed over time? Provide examples.] 


a. To what extent are Palms used specifically to implement SR frameworks, strategies and 
instructional methods? 
[Probe: How are students using them (e.g. completing projects)? How are teachers using 

them (e.g. designing lessons, reviewing student work, assessment) ?] 

b. To what extent are Palms used to monitor student performance or teacher effectiveness in 
literacy instruction? Please describe these processes. 

c. What evidence is there that use of the Palms increases student motivation and engagement in 
these activities? 
What evidence is there that use of the Palms improves student learning? 

e. What evidence is there of teachers’ and/or LITs’ preparedness and comfort level using the 
Palms for these activities? What specific factors facilitate or hinder staffs use of the Palms? 

f. What evidence is there of school level administrative support for use of the Palms? What 
form does this support take, where it exists? 


6. What is the status of the following features of the Palm Pilots? 
a. Teachers and students uploading their usage data to the server. How is this feature being 
used to further augment literacy instruction? 
b. Automatically syncing Palms with desktop computer 
c. Using the palms for wireless internet access for teachers and students 
d. Designing more usable literacy based lesson plans that integrate handhelds or desktops in 
small group instruction 


7. In what ways is the Chicago Striving Readers program currently using each of the following 
additional types of technology as a tool to help improve differentiated literacy instruction for 
readers at all levels? [Probe: If/how other technologies are used to monitor student performance or 
teacher effectiveness in literacy instruction.]What have been the successes and challenges of these 
efforts? How have these changed over time? 

e Classroom media centers’ 
e Listening centers* 


8. What does theStriving Readers program need to better support the use of technology to 
improve literacy instruction? 


9. Is there anything else you would like to add regarding the program or literacy activities for 
struggling readers in the district? 


73 computers and a printer 
‘students access models of fluency, record and listen to themselves reading,and listen to audio books 
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Observation Protocol Form(from Excel) 


Instructions: 


The Observation data will be submitted to Metis Associates in this Excel database, 
which has been pre-formatted for each section of the Observation protocol (as noted 
by the worksheet tabs at the bottom of the spreadsheet). In order to facilitate the data 
analysis process, please ensure that the following steps are followed when you enter 
and submit your data: 


>> Please re-name the Excel file with the name of the school and the date of the 
observation. 


>> Be sure that the identifying information for each observation is completed on 
the top of each worksheet tab, including Observer, Date, Time of Lesson, School, 
Classroom Number, Teacher Name, and Observation Day (whether this was the first or 
second day of observation for this class). You may copy and paste these data from 
tab to tab. 


>> For the items in the protocol that have check boxes, a drop-down list has been 
provided for you to select your response. There is a note indicating this within the 
spreadsheet in blue font. Please be sure that a selection is made for each item. 


>> For the Striving Readers frameworks, strategies, and techniques that are 
covered in the lesson (as indicated during the pre-observation interview and Part IV 
Specific Striving Readers Frameworks, Strategies and Techniques of the Observation 
protocol), please make sure that you complete the corresponding worksheet tab. For 
example, if the teacher implemented Small Group Instruction, you will need to provide 
evidence of the instruction on the ‘Small Group Instruction’ tab. 


>> With the exception of the drop-down list responses, all of the cells in the Excel 
database will allow for over 32,000 characters, which equates to more than six pages 
of text in Word. It is possible that you will write more than is visible in the Excel field. 
Please rest assured that your text has not been truncated. 


It is very important that you do not modify this structure. Please 
do not add or remove rows, columns, or worksheets. 


If you have any questions or problems using this database, please contact Rebecca 
Swann-Jackson at 212-430-9113 or rswann@metisassoc.com. 
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Pre-Observation Interview 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop- 
down list) 


Primary Goals of the Lesson 


Specific Skills/Knowledge to be Developed 


Formal/Informal Assessment 


Other Adults 


SR Frameworks 


Whole Part Whole 


Independent Reading 


Small Group Instruction 


Intervention 


Reading Comprehension Strategies 


Summarizing 


Questioning 


Predicting 


Visualization 


Text structure 


Inferring 


Metacognition 


Response (Choose response 
from the drop down list) 


Response (Choose response 
from the drop down list 
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Reading Comprehension Techniques 


Marzano's Vocabulary 


PRC2 


Word Study/Word Sorting 


Interactive Read Aloud 


Reading Response 


INSERT Notes 


PLAN 


ReQuest 


KWL 


List-group-label 


Other 1 (Select "Checked" from the drop down list to the 
right, and specify the technique in the blue cell below it) 


Response (Choose response 
from the drop down list) 


Other 2 (Select "Checked" from the drop down list to the 
right, and specify the technique in the blue cell below it) 


Other 3 (Select "Checked" from the drop down list to the 
right, and specify the technique in the blue cell below it) 


AMPAfter-School Program 


Response (Choose response 
from the drop down list) 


Achieving Maximum Potential (AMP) Program 
Any Additional Notes for Question 3 Response 


If there are any additional comments on the items in the 
lists above (Question 3), please enter them here. 


Grouping Criteria Response (Choose response 
from the drop down list) 


Similar groups by tier/reading ability 


Mixed groups by tier/reading ability 


Similar groups by other special needs 


Mixed groups by other special needs 


Students’ preference 


Randomly assigned 


Other 1 (Select "Checked" from the drop down list to the 
right, and specify the technique in the blue cell below it) 


Response (Choose response 
from the drop down list 


Different grouping criteria 
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Response (Choose response 
from the drop down list) 


Textbook or novel 


Text set small books 


Classroom libraries 


Vocabulary or reading response notebooks 


AMP materials 


Academic vocabulary notebooks (Marzano) 


PRC2 folders 


Graphic Organizers 


Word Sorting/Words their Way materials and notebooks 


Writing folder or notebook 


Trade novels 


Basals 


Materials for Handheld Computers (Palm Pilots) 


Other 1 (Select "Checked" from the drop down list to the 


right, and specify the technique in the blue cell below it) 


Other 2 (Select "Checked" from the drop down list to the 


right, and specify the technique in the blue cell below it) 


Any Additional Notes for Question 4 Response 


If there are any additional comments on the items in the 
lists above (Question 4), please enter them here. 


Instructional Technology Response (Choose response 
from the drop down list) 


Media centers 


Listening centers 


Word processing 


Spreadsheets 


Internet 


Digital Media (e.g., camera, video, etc.) 


Handheld Computers (Palm Pilots) 


AMP Software 


Other 1 (Select "Checked" from the drop down list to the 
right, and specify the technique in the blue cell below it) 


Other 2 (Select "Checked" from the drop down list to the 


right, and specify the technique in the blue cell below it) 


Any Additional Notes for Question 5 Response 


If there are any additional comments on the items in the 
lists above (Question 5), please enter them here. 
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Brief Summary 


Context of today's lesson 


Approximate # in Tier 1 
Approximate # in Tier 2 
Approximate # in Tier 3 


Special Needs? 
ELLs? 


Anything else | should know? 


Post-Observation Interview 


Observer 
Date 
Time of Lesson 
School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down list) 


Results/Surprises 
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Lesson Changes 


Clarifying Information 


Questions/Concerns 


Observation Protocol Part I 


Observer 


Date 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 


Subject (Choose from drop-down list) 


Grade (Choose from drop-down list) 


Class Session Start Time 


Class Session End Time 


Observation Start Time 


Observation End Time 


Number of Male Students 


Number of Female Students 


Adults in the Room 


Response (Choose from drop-down 
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list) 


Classroom Teacher 


LIT 


Other 1 (Select "Checked" from the drop down list 


to the right, and specify the technique in the blue 
cell below it) 


Submit the Classroom Sketch separately. 


**Please remember to take detailed field notes during the observation and to 
complete the remainder of this protocol only after leaving the classroom.*** 


Please provide a brief narrative account 

summarizing the lesson as observed and 

describe each individual activity in the Response 
sequence in which they occurred (include 

concurrent small group activities as 

separate activities). 


Narrative Account 


Be sure to address the following issues for each 
activity identified: 

the timeframe (approximate start and end times ) 
and sequence in which they occur; 

* grouping structure(s) (whole class, small group, 
partners, individual work) 

¢ the numbers of students involved (specify whether 
more than one small group is working on the same 
activity) 

¢ whether the teacher, LIT and/or other adults are 
involved and their roles (lecturing, modeling, 
explaining, etc.) 

¢ format (presentation, discussion, silent reading, 
etc.) 

* types of interactions (student-student, student- 
teacher, LIT-teacher, LIT-student, etc.) 


Submit the Lesson Plan and any Handouts separately. 
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II. Classroom Data 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down list) 


Complete the Classroom Physical Characteristics/Management and Affective 
Quality section below, and the General Notes section that follows, based on your 
entire observation. Provide details as appropriate when answering questions if 
the responses differ by activity. 


Check yes or no for each question and provide Response (Choose 
clarifying information as appropriate. response from the drop 
down list) 


Is the daily schedule posted? 


Is student authentic work is posted? 


Are there are student/teacher created charts? 


Is there a listening center? 


Is there a media center? 


Are there desktop or laptop computers? 


CO} N) } OI) RB} OO) NM) 


s there classroom library? al 


Are there handheld computers? 


Any Additional Notes for Part II 


If there are any additional comments on the items in the lists 
above, please enter them here. 
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What evidence is there that... Response 


1. Resource material related to the activities is 
accessible to groups and/or individual students? 


2. The adult is positioned so all students can easily 
view modeling and/or materials being introduced and 
used during instruction? 


3. There are sufficient books and materials for 
independent, paired, or grouped student work? 


4. There are there high interest and varied (reading 
level) reading materials for students? 


5. There is asense of order and routine? 


6. There is a respectful atmosphere? 


7. There is a purposeful interaction taking place 
between teacher and student or student and student? 


8. There are shared expectations for learning and 
achievement? 


9. Students understand the importance of content? 


10. Students collaborate and construct knowledge in 
respectful and responsive ways? 


11. Students and teachers use language that shows 


respect like “thank you”, “that was a good idea’, “I 
hadn’t thought of that”, “that is interesting’. 


12. The classroom creates a safe environment for 
expressing ideas? 


13. The teacher and students embrace cultural 
differences and honor each student’s identity? 


14. Students are interacting with peers, teachers and 
LITs? 


15. Students are discussing or talking about the 
content of learning activities (reading/writing)? 


16. The physical and emotional environment support 
learning? 
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III. General Notes 


Observer 
Date 
Time of Lesson 
School 
Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down list) 


Complete the General Notes section based on your entire observation. Provide details 
as appropriate when answering questions if the responses differ by activity. 


1. The teacher/LIT is helping students develop reading comprehension strategies, including: 


summarizing 


questioning 


predicting 


visualization 


text structure 


inferring 


metacognition 
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2. The teacher/LIT provides reading comprehension 
instruction for whole class with blended intervention 
(small group guided practice, differentiated work)? 


3. The teacher is engaged with instruction (not 
grading papers or otherwise occupied)? 


4. The teacher/LIT moves from whole class 
introduction with explicit instruction to small group 
work so students can try out the strategies or 
questioning routines in materials and with other 
students working at their own level? 


5. The teacher/LIT guides students toward reading 
materials that are individualized for different students 
(differentiated instruction)? 


6. The teacher/LIT activates prior knowledge and/or 
builds background knowledge with students? 


7. The teacher/LIT is assessing students during 
instruction formally or informally? 


8. The teacher and LIT work together to differentiate 
instruction? 


9. The teacher/LIT encourage students to make 
connections about the activity to other outside 
selections and/or to make personal connections? 


10. The teacher/LIT models the specific strategy/skill 
to be employed during the activity? 


11. The teacher/LIT introduce vocabulary and 
concepts in context and/or discussion? 


12. The teacher/LIT use writing models (or 
exemplars) to stimulate student thinking? 


13. The teacher/LIT provide explicit guided instruction 
at various times during the lesson? 


14. The teacher/LIT use different modes of activities 
(e.g. partnered activities, teacher guided instruction, 
independent activities) to meet individual student 
needs? 
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15. The teacher/LIT or other adults meet the needs of 
special needs students who require additional support 
by incorporating additional strategies (is there a 
Special Education teacher available...an aide)? 

16. The teacher/LIT assess students’ understanding 
of the lesson and task directions? 


17. The teacher/LIT provide students with 
opportunities to demonstrate learning outcomes 
(assessments)? 


18. The teacher/LIT provide feedback to students on 
progress made? 


19. The teacher/LIT encourage students to work with 
technology? 


20. The teacher/LIT use scaffolding to help students 
understand content. 


21. Students show sustained reading when reading 
independently? 


IV. Specific SR Frameworks, etc. 


Observer 
Date 
Time of Lesson 
School 
Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 
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SR Frameworks 


Whole Part Whole 


Independent Reading 


Small Group Instruction 


Intervention 
Reading Comprehension Strategies 


Summarizing 


Questioning 


Predicting 


Visualization 


Text structure 


Inferring 


Metacognition 
Reading Comprehension Techniques 


Marzano's Vocabulary 


PRC2 


Word Study/Word Sorting 


Interactive Read Aloud 


Reading Response 


INSERT Notes 


PLAN 


ReQuest 


KWL 


List-group-label 


Other 1 (Select "Checked" from the drop down list 
to the right, and specify the technique in the blue 
cell below it) 


Response (Choose response 
from the drop down list 


Response (Choose response 
from the drop down list) 


Response (Choose response 
from the drop down list 


Other 2 (Select "Checked" from the drop down list 
to the right, and specify the technique in the blue 
cell below it) 


Other 3 (Select "Checked" from the drop down list 
to the right, and specify the technique in the blue 
cell below it) 


AMPAfter-School Program 


Response (Choose response 
from the drop down list) 


Achieving Maximum Potential (AMP) Program 
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Whole-Part-Whole 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 


Describe the Whole - Part - Whole Whole-Part-Whole Response 
Activities 


1. What are the teacher(s) and students doing 
during the whole class activity? 


2. What are the teacher(s) and students doing 
during the small group/pairs activity? Describe 


the grouping structures. 


3. What are the teacher(s) and students doing 
during the final whole class activity? 


What evidence is there that... Whole-Part-Whole Response 


1. The teacher has organized the class such 
that there is whole group instruction at the 
beginning of an activity? 


2. The teacher has small group or individual 
activities for students (following) the whole 
group piece of the lesson? 


3. The teacher facilitates whole group 
instruction or follow-up after the small 
group/individual activities? 


4. The teacher has clear guidelines for 
students regarding the organization of 
individual/small group activities? 


5. Students move easily between whole 
group and individual/small group instruction 
and seem comfortable with this organization 
of class activities? 
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Independent Reading 


Observer 
Date 
Time of Lesson 
School 


Classroom Number 


Teacher Name 
Observation for this class (Choose from drop-down 
list) 


What evidence is there that... Independent Reading Response 


1. The teacher has selected a topic for mini- 
lesson based on student needs/interests and 
curriculum goals? 


2. The teacher monitors and analyzes 
students’ reading records? 


3. The teacher confers with students during 
independent reading session to teach and 
reinforce skills and strategies? 


4. The teacher assesses students using 
running records? 


5. The teacher assists students with the 
routines of independent reading? 


6. The teacher gives book talks to heighten 
engagement and motivation for reading? 


7. The teacher concludes each session with 
whole group sharing that evaluates 
independent reading session and ties student 
reading back to mini-lesson focus? 


8. The teacher establishes procedures and 
routines for independent reading? 


9. The teacher schedules a consistent time 
for independent reading? 


10. The LIT assists in identifying students’ 
independent reading level? 
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11. The LIT administers formative 
assessments for Tier Three students during 
independent reading time? 


12. The LIT guide students in selecting 
appropriate texts for independent reading? 


13. Students read accessible texts. 


14. Students are actively engaged in reading 
and in responding to what they have read. 


15. The classroom library that contains a 
variety of genres and a topics and titles at 
various reading levels, including graphic 
novels, magazines, newspapers, and other 
media. 


16. Student self-selected reading materials 
and response notebooks conveniently 
housed. 


17. Teacher actively engages students in 
conversation about books in which reading 
strategies and student self-evaluation of 
independent reading process are included. 


Small Group Instruction 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 
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What evidence is there that... Small Group Instruction 
Response 


1. There is a sufficient quantity of leveled 
narrative and expository texts matched to the 
students’ independent and instructional 
levels? 


2. The teacher/LIT introduces the text, 
consistent with the meaning, language, and 
visual information in the text, and the 
knowledge, experience, and skills of the 
reader. 


3. The teacher/LIT interacts with students 
individually observing strategy use, difficulties 
and successes with problem-solving 
attempts? 


4. The teacher/LIT returns to the text for one 
or two teaching opportunities to demonstrate 
how a reader constructs meaning from text, 
makes personal connections with text, and 
goes beyond text? 


5. Students engage in pre-reading 
conversation about text? 


6. Students read a text or part of a text 
silently or quietly? 


7. Students request problem-solving help 
when needed? 


8. Students revisit the text at points of 
problem-solving as guided by the teacher? 


9. Students engage in collaborative 
discussion about the text? 


10. Students engage in activities that involve 
extending understanding and responding to 
the text? 


11. There are a variety of instructional 
practices used to meet individual students’ 
needs. 


12. Students are grouped flexibly, including 
homogeneous and heterogeneous groups for 
different reasons at different times? 


13. The teacher assesses literacy 
performance in a variety of ways over time, 
including checklist observations of student 
literacy behaviors? 
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Intervention 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 


What evidence is there that... Intervention Response 


1. The teacher models what expert readers 
do by modeling before, during, and after 
reading literacy strategies? 


2. The teacher/Interventionist (LIT) explicitly 
teach literacy strategies and include the 
supporting skills of vocabulary to support word 
study in context? 


3. The teacher/Interventionist (LIT) provide 
whole group instruction, incorporating read- 
alouds. 


4. The teacher/Interventionsit (LIT) provide 
small group instruction by bringing students 
with similar needs together and giving them 
attention tailored to those needs? 


5. The teacher provides opportunities for 
partners/pairs to have time to think through 
their reading? 


6. The Interventionist (LIT) assesses 
student needs? 


7. The Interventionist (LIT) assesses, 
teaches/reteaches, practices, applies, and 
reassesses. 


8. Students self-monitor their reading and 
select strategies needed for comprehension. 


9. Students participate with others in their 
differentiated groups. 


10. The interventionist (LIT) is working one- 
on-one with a student. 
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11. The interventionist (LIT) is weaving in and 
out of students’ activities. 


12. There are flexible grouping of students 
relative to student needs and types of class 
activities. 


13. The classroom seems to be a 
collaborative setting where students monitor 
and discuss their own progress. 


Marzano’s Vocab 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 


Marzano's Vocabulary 
Response 


What evidence is there that... 


1. The teacher has established routines, 
rituals, and expectations to students around 
the 6 step process and are using the 
academic vocabulary notebooks? 


2. The teacher follows the 6 step process to 
ensure systematic, direct vocabulary 
instruction? (See six step process below and 
reflect on its usage in this observation) 


Step 1: Provide a description, 
explanation, or example of the new term. 


Step 2: Ask students to restate the 
description, explanation, or example in their 
own terms. 


Step 3: Ask students to construct a 
picture, symbol, or graphic representing the 
term or phrase. 
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Step 4: Engage students periodically in 
activities that help add to their knowledge in 
their notebooks. 


Step 5: Periodically ask students to 
discuss terms with one another. 


Step 6: Involve students periodically in 
games that allow them to play with terms. 


3. Students utilize the 6 step process in 
order to build knowledge of academic content 
vocabulary? 


4. Students use their academic vocabulary 
notebooks to record and monitor 
understanding of content concepts? 


5. There is direct instruction on words that 
are critical to content concepts? 


6. Students are engaged in writing and 
drawing about words and concepts in their 
notebooks? 


7. Teachers and students are using a 
variety of methods to deepen and extend 
understanding of terms (e.g. comparing, 
classifying, creating metaphors, creating 
analogies)? 

8. Teachers and students are involved in 
review activities and games, such as 
Jeopardy!, vocabulary charades, $100,000 
Pyramid, etc? 


PRC2 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 
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What evidence is there that... 


1. Classrooms have inviting materials at a 
range of reading levels and the PRC2 routine 
clearly established? 


PRC2 Response 


2. Classrooms have students productively 
engaged with partners comfortably using the 
PRC2 framework? 


3. All students are reading informational 
materials on a common theme or topic at their 
independent or instructional level? 


4. Students discuss answers to questions 
they have asked each other and use the texts 
as resources? 


5. Teachers move around the room 
observing some partners in depth - listening to 
students as they read and discuss and make 
notes about their performance? 


6. Students keep a record of pages they 
read with questions and responses, and new 
vocabulary? 


7. Opportunities exist for students in 
differentiated practice using features of 
informational texts (structure and visual 
information), learning new vocabulary 
(morphology and concept clusters), 
developing fluency, and thinking critically by 
comparing and contrasting texts? 


8. Classroom is a learner-centered 
environment with opportunities for sharing 
across texts and learning experiences? 


9. There is an established routine in the 
classroom so students know exactly how to 
locate their materials, engage productively 
with their partner, and keep record of their 
PRC2 sessions? 


10. The teacher models how to preview texts 
and attend to the Table of Contents, text 
resources (glossary, index, and on-line 
references), external features (headings, sub- 
headings, highlighted vocabulary and visual 
information) and the internal structure. 


11. The teacher models for students the 
process of preparing for, reading and 
discussing the content with a partner? 
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12. The teacher or LIT monitors students 
during the time they are engaged in PRC2 
and keep notes on the sessions? 


13. The teacher or LIT provides positive 
feedback to students about their partner 
process? 


14. The teacher presents “mini-lessons” to 
refresh the process if students get off track? 


15. The teacher leads discussions at the 
conclusion of PRC2 sessions in which 
students can share information they have 
learned, connections they have made, and 
questions they have? 

16. Students prepare for each page by 
reading silently, thinking about oral prosody, 
and identifying unfamiliar words so they can 
be pronounced accurately and then recorded 
in the vocabulary notebook so they can be 
learned over time? 

17. Students select or create questions to ask 
one’s partner? 


18. Students listen attentively as the partner 
reads orally and respond to the partner’s 
questions? 


19. Students read designated pages orally 


with fluency? 


20. Students participate in discussion 
courteously and with interest? 


Word Sorting Study 


Observer 


Date 

Time of Lesson 
School 

Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 
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What evidence is there that... Word Sorting/Word Study 
Response 


1. The teacher selects word sorts at the 
students’ instructional level that will support 
and scaffold their word knowledge? 


2. The teacher introduces new sorts to 
students by modeling and then guiding their 
work. 


3. The teacher interacts with students 
individually and in small groups to guide them 
and assist in problem solving? 


4. The teacher monitors word study through 
observation, note-taking, and targeted 
instruction? 


5. The teacher creates a supportive 
environment for word learning? 


6. Students engage in word sorts 
independently and in small groups? 


7. Students engage in collaborative 
discourse with other students? 


8. Students engage in activities that will 
extend understanding of the word parts, word 
meanings, and general and specific word 
knowledge? 


9. Students request problem-solving help 
when needed? 


10. The teacher circulates the classroom 
assisting students and asking guided 
questions? 


11. The teacher is engaged in targeted, small 
group instruction with students? 


12. The students explain their thinking to the 
teacher and the students? 


13. Students use their Palms during this 
activity? 
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Interactive Read Aloud 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 


What evidence is there that... Interactive Read Aloud 
Response 


1. The teacher shows an enthusiasm for 
reading and communicates interest in 
reading? 


2. The teacher shares books and articles 
with students and to model reading behavior? 


3. The teacher models persistence and 


stamina for reading long and sometimes 
difficult text? 


4. The teacher occasionally stops to define 
a word, to wonder aloud, to ask questions, 
and to respond to the reading? 


5. The teacher gives students opportunities 
to respond to the reading by discussing and 
writing about the text? 


6. The teacher has chart paper, overhead 
transparencies, and markers to record student 
questions, comments, and ideas, especially if 
the same text will be used for several days? 


7. The teacher provides a format for 
students to keep a record of the read alouds 
in order to remember authors, genres, and 
topics for further reading and investigation? 


8. The teacher gives clear expectations of 
student behavior during the read aloud? 


9. Students seem to enjoy listening and to 
discuss the read aloud? 
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10. Students explain, question, and explore 
ideas in what they have heard? 


11. Students keep a record of the read aloud 
along with responses and reactions? 


12. The teacher is respectful of all students’ 
cultures and backgrounds. 


Reading Response 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 


What evidence is there that... ‘Reading Response’ Response 


1. Teacher creates a safe environment 
where students are encouraged and 
respected? 


2. Teacher has established routines for 
Reading Response Notebook/Learning Log 
usage and record-keeping? 


3. Teacher explicitly instructs, models, 
scaffolds, and coaches skill lessons for writing 
different types of responses and for talking 
about literary elements? 


4. Teacher uses read-alouds to generate 
and model discussion? 
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5. Teacher scaffolds students toward 
accountable talk and establishes routines for 
conversation (e.g., Staying on topic, 
discussion etiquette, adherence to the text, 
active listening, and including everyone in 
conversation)? 


6. The students demonstrate understanding 
of a text through written response and 
classroom discussions? 


7. The students share thoughts and 
opinions during whole group and small group 
discussion? 


8. The students make a judgment and 
support it with specific references to the text 
and background knowledge? 


9. The students actively listen to classmates 
and contribute to the conversation 
collaboratively? 


10. The teacher provides opportunities to use 
writing to respond in all content areas? 


11. The teacher encourages participation and 
collaborative discussion based on the writing 
students have completed. 


12. There is conferencing between the 
teacher and individual or small group of 
students. 


INSERT Notes 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 
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What evidence is there that... INSERT Notes Response 


1. The teacher is leading a whole group 
demonstration of the INSERT note strategy or 
students independently using the strategy with 
independently read text. 


2. The teacher describes the INSERT notes 
strategy and why it is helpful. 


3. The teacher selects a piece of text to 
demonstrate the strategy 


4. The teacher thinks aloud the reasons for 
choosing a particular symbol. 


5. The teacher elicits student responses on 
which symbol to use and why. 


6. The LIT demonstrates INSERT note 
strategy and work intensively with individual or 
small groups of Tier 2 and Tier 3 students. 


7. Students use the strategy with a common 
text in pairs or teams. 


8. With common texts, students compare 
and discuss their INSERT notes. 


9. With independently read text, students 
write notes reflecting their thoughts, questions 
and comments directly on photocopied text, 
on 4 column charts, or on post-it notes which 
are inserted onto text. 


Predict-Locate-Add-Note 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 
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What evidence is there that... Predict-Locate-Add-Note 
PLAN) Response 


1. Students create a graphic organizer 
showing the major sections of the text being 
used as their reading materials (textbook, 
magazine article, etc.). Then students use 
this organizer to make notes as they read. 


2. Students write a summary of their notes 
and the major points from the graphic 
organizer at the conclusion of the summary. 


3. The teacher/LIT explains the process and 
models if this is the first use. 


4. The teacher/LIT supports students as they 
create their own organizers. 


ReQuest 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down 
list) 


What evidence is there that... 


ReQuest Response 


1. Teacher and LIT modeling and providing 
feedback in small group activity 


2. Teacher, LIT and students engaged in 
discussion about segments of text stemmed from 
questions developed by teacher, LIT or student 


3. Students applying strategy in whole class 
and small group instruction 
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4. The teacher and LIT choose a passage of 
text, then designate short segments within the 
passage 


5. The teacher and LIT provide an example 
through modeling and feedback 


6. The teacher and LIT conducts first round of 
ReQuest activity so that he/she is the one to 
answer questions generated by the students 


7. The teacher and LIT keeps book closed 
during the questioning 


8. The teacher and LIT tailor ReQuest to suit 
the specific needs of students 


9. The teacher and LIT provide task cards to 
struggling readers with specific questions from 
text segments 


10. The teacher and LIT scaffold questioning and 
answering process to struggling readers (Tier 2 & 
Tier 3 students) 


11. Students watch carefully for teacher’s 
modeling and feedback 


12. Students read passage silently. Pay attention 
to the information it contains. 


13. Students think of questions they may be 
asked if they were the respondent. 


14. Students use own words and check passage 
for possible answers 


15. Students keep book open while asking 
question. 


16. Students keep book closed while they 
answering question. 


17. Students listen to answers and check in text 
for accuracy. 


18. Students change roles — take turns in being 
questioner or respondent 


19. Students apply strategy during whole class 
or small group discussion 


KWL 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down list) 


What evidence is there that... KWL Response 


1. Teacher is leading a large group introductory 
discussion of the topic to be studied and writing 
down the student contributions. 


2. Teacher stimulates disagreement and helps 
focus questions for inquiry. 


3. Students are engaged in listening to each 
other and sharing what they know and their 
questions. 


4. The teacher models active comprehension 
by helping students activate their prior 
knowledge and formulate questions. 


5. Teacher is scribe and records contributions. 


6. Teacher elicits questions and 
disagreements. 


7. The LIT sits with and assists Tier 2 & 3 
students during large group discussion. 


8. The LIT helps students write their own KWL 
notes or enter them on their Palm. . 


9. The LIT may discuss part of the text with 
students so they can access difficult sections. 


10. Students listen to each other 


11. Students contribute ideas and questions 


12. Students use their own KWL sheets to record 
their ideas and personal questions. As they read 
the text they make notes of what they learn. 


13. Students use their Palms during this activity. 
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List-Group-Label 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down list) 


What evidence is there that... 


1. Small groups of students are working 
together to generate a list of words related toa 
content area topic and then grouping the words 
into logical categories. This activity will take 15 to 
20 minutes with the teacher and LIT working with 
the small groups. 


List-Group-Label Response 


2. The teacher selects a main topic or concept 
from a content area reading selection and models 
the process by brainstorming and charting with 
the class words that are related to the topic. 


3. The LIT is working intensely with the tier 2 & 
3 students to support the grouping and labeling 
process. 


4. Students are working in teams of three to 
four, students join together related terms from the 
brainstormed list based on common features. 


5. Students generate a descriptive title, or label 
for the collections of related terms. 


6. Following the reading of the pre-selected 
content area text, students eliminate any of the 
terms or groups that do not match the concept’s 
meaning in the context of the selection or add 
new terms or groups as needed. 
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AMP 


Observer 


Date 


Time of Lesson 


School 


Classroom Number 


Teacher Name 


Observation for this class (Choose from drop-down list) 


What evidence is there that... AMP Response 


1. The teacher provides strategic tutoring? 


2. The teacher teaches one comprehension 
strategy at a time directly and explicitly? 


3. The teacher teaches strategies intensely and 
systematically? 


4. The teacher provides intensive writing 
opportunities? 


5. The teacher provides successful learning by 
delivering the direct instruction, practice, and 
evaluation students need to achieve maximum 
success? 


6. The teacher utilizes the technology 
component? 


7. Students seem to give full attention and 
effort? 


8. Students demonstrate a cooperative learning 
attitude toward fellow students and teacher? 


9. Students stay on task and follow directions? 


10. Students are engaged with reading 
strategies? 


11. The teacher uses small group and 
differentiated instruction. 
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CPS Striving Readers 
Case Study — Fall 2008 School Visits 
Self Contained Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. Your school was among six schools that were selected for an 
in-depth case study in part because they were identified by the district as being successful in their 
implementation of the Striving Readers initiative. We are interested in learning about the implementation 
of this program in your school, overall and for each program component, identifying best practices, and 
gaining a better understanding of the facilitating conditions and challenges to implementation. Since we 
are requesting a lot of information and I know you have a busy schedule, please be as succinct as you can 
in your answers. You will have an opportunity to elaborate further at the end of the focus group. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 
recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 
specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to yourself or your school. Is this all right? 


1. Please introduce yourselves and tell us how long have you been in this school, and which grade 
level(s) you teach this year. 


2. We would like to know about your use of specificStriving Readers instructional 
frameworks, techniques and strategies. 


a. How comfortable are you with the different frameworks, techniques and strategies identified 
in the Striving Readers grant? When did you start using them? 


b. Which oneshave you found to be the most successful and why? Which ones have presented 
the greatest challenges and why? 


Probe and request examples: 


o Striving Readersinstructional frameworks (for example: whole part whole 


instruction, differentiating instruction, independent reading, small group 
instruction) 


oO Core Comprehension Strategies(Summarizing, Visualizing, Questioning, 


Metacognition, Inferring, Predicting, Text structure) 
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o Core Comprehension Techniques (for example: Marzano’s vocabulary, PRC2, 
KWL, Word Study/word sorting, Read Aloud/Think Aloud, Reading Response, 
INSERT notes, PLAN, ReQuest, List-Group-Label, etc.) 


3. We would like to know more about the process that you, teachers and students use to select 
appropriate student reading materials. 


a. Do you use fluency snapshots and interest inventories to guide students’ selection of 
reading materials for their wide reading? Please provide examples. 


b. Are student reading levels used to help select appropriate materials? Please explain and 
provide examples. 


c. To what extent do students know how to select materials for wide reading that are 
appropriate to their abilities? Please explain and provide examples. 


Next, I would like to ask you a few questions about the specifics of when you meet with other 
teachers to plan and coordinate instruction and student groupings for the purpose of providing 
differentiated instruction that is appropriate to each student’s needs. 

4. Does your school have Grade-Level Teams? Yes UNo (IF NO, SKIP TO Q5) 


a. Are you all involved in these teams? 


b. What is your role in these teams? What topics are discussed in these meetings? 


5. Does your school have a Literacy Leadership Team? (Yes (No (IF NO, SKIP TO Q6) 
a. Are any of you involved in this team? (Yes (No (IF NO, SKIP TO Q6) 


b. What is your role in this team? What topics are discussed in these meetings? 


6. We would like toknow more about your collaboration with the Literacy Intervention Teacher 
and how you plan together to provide blended, differentiated instruction. 


a. In which setting(s) do you meet or collaborate with the LIT? 
O One-on-one meetings: 
Oo Grade-level team meetings: 
oO Literacy leadership team meetings: 


b. Do you collaborate with the LIT in scaffolding instruction (comprehension, vocabulary 
and fluency)? Please provide examples. 


c. Do you plan together with the LIT for blended instruction that differentiates small group 
work? Please provide examples. 
0 Do you collaborate with the LIT to use assessment or other data to fine-tune 
differentiated instruction? Please provide examples. 


d. Please describe other topics you discuss when meeting with the LIT to ensure that Tier 2 


and 3 students receive appropriately targeted, differentiated instruction during in-class 
targeted intervention sessions and AMP after-school lessons. 
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e. Have there been any changes in the strategies, activities and resources that you use during 
these push-in sessions, as compared to last year? 
oO If yes, please describe. 
Oo Why did these changes come about? 


f. What challenges, if any, have you encountered while working with the LIT and/or Tier 2 
and 3 students this year? 
Oo How were or will these challenges be addressed? 
7. We would like to learn more about your collaboration with the District Coordinator. 
a. Do you meet with him/her? If so, how often do you meet with him/her? 
__Oncea ___ Several ____ Once ___ Several ___ Daily or 


month or less times a month a week times a week almost daily 


b. In what ways do you collaborate? What topics do you discuss? In what ways does 
he/she support your implementation of Striving Readers? 


c. Is there any additional support they could provide you with? 
8. In what other ways, on your own or in collaboration with the LIT, do you use student data? 
(This might include assessment data as well as other types of student data such as demographic 
and behavioral records.) 


a. Which kinds of data are you using? 


b. In which setting(s) (e.g., individually, as part of grade-level teams, as part of the Literacy 
Leadership Team, other) 


c. For what purposes? Please describe and provide examples. 
Probe & request examples: 

oO Plan for small group activities, 

O Monitor students’ success in learning techniques for developing comprehension and using 
reading strategies appropriately, 


oO Select appropriate materials at students' independent and instructional levels. 


We would like to know more about your efforts, if any, to integrate literacy into other content area 
instruction. 


9. When, if at all,did you start integrating literacy into your content area instruction? (Probe for: 
this year, last year, before Striving Readers began?) 


10. Have you been able to use any of the specific frameworks, techniques and strategies identified 
in the Striving Readers grant? 


a. Ifnot, how have you approached integrating literacy in your classroom? 


b. Ifyou have used specific Striving Readers frameworks, techniques and strategies, please 
provide examples. 
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11. We would also like to know more about the types of support that you have received to help you 
integrate literacy instruction into other content areas. 


Have you met with any literacy experts to specifically discuss the integration of literacy 
into other content area instruction? How often? What did you discuss with them? [Probe 
for: Literacy Intervention Teacher, Lead Literacy Teacher/Literacy Coach, Striving 
Readers Coordinator, other] 


Have you ever participated in any professional development conducted by the Striving 
Readers program related to integrating literacy into other content area instruction? 
LYes LNo 


If yes, when/what year did you start participating in this type of trainings? 

Which trainings did you participate in? (Probe and request specific examples: site-based 
professional development, Summer Institute and follow-up Institutes, Saturday Seminars) 
Which trainings were most useful and why? Which trainings were least useful and why? 
What areas or topics would you like to receive additional support or training in? 


Does your school have school-wide text sets (i.e., sets of reading materials of different 
structures and levels, centered around specific content area themes, designed to improve 
student literacy in other subject area classes)? Yes c No 


If yes, are the school-wide text sets being used in the content area classrooms? 


oO Social Studies 5 Not Used—why? o Used—how? 0 Don’t Know 
0 Mathematics o Not Used—why? o Used—how? o Don’t Know 
oO Science 0 Not Used—why? 0 Used—how? o Don’t Know 


What other kinds of support areavailable to help you integrate literacy into your content 
area instruction? 


What other kinds of support would you need to help you integrate literacy into your 
content area instruction? 


We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 


12. One of the components of Striving Readers is building your classroom libraries. 


a. 


How do you use the classroom libraries? 


Probe & request examples: 


For independent reading? 

For small group instruction? 

To support content area instruction? 
For read alouds? 


A-343 


b. Do you use interest inventories to help students self select reading material and to guide 
your purchases? Please describe the process. 


c. How do you organize books in your classroom library? (Probe: Is there more than one 
criterion used to organize the libraries?) 


d. Have you or your students encountered any challenges in using your classroom libraries 
to support instruction? If so, please describe. 
oO How have or will these challenges be addressed? 


13. Does your school have Listening Centers (where students can access models of fluency and 
record themselves to assess their own fluency)? [Yes [No (IF NO SKIP TO Q14) 


a. 


Where are these Listening Centers located? 

Li In the classrooms only 

(| Outside of the classrooms only (e.g., computer lab) 
Oo How accessible are they? 

11 Both 


Do you use the Listening Centers? [Yes (INo 
(if not used) Why not? 


(if used) How are you using them? 
fe) For which type of activities? 
O Are you using the Listening Centers with all students or subgroups of these 
students? 
fe) Please provide an example of how you use the Listening Centers to help 
differentiate instruction. 


(If used) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 
Oo How were or will these challenges be addressed? 


14. Do you have Media Centers (3 computers and 1 printer) in your classrooms)? (|Yes_ (INo (IF 
NO SKIP TO Q15) 


a. 


b. 


Do you use the Media Centers? Yes UNo 
(If not used) Why not? 


(If used) How are you using them? 
O For which type of activities? 
O Are you using the Media Centers with all students or subgroups of these students? 
O Please provide an example of how you use the Media Centers to help differentiate 
instruction. 


(If used) Have you or your students encountered any challenges when using this technology? 


If so, please describe. 
oO How were or will these challenges be addressed? 
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15. Do you and/or students in your school have access to Palm Pilots/Handheld Computers? 
(Yes (| No> Ifno, why not? (THEN SKIP TO Q16) 


a. Do you and/or your students use them in your classroom? [1Yes LINo 
b. (If not used either by Teachers or by Students) Why not? 
c. (fused), how are they being used? 
oO For which type of activities do you use the Handheld Computers? 
o Are the Handheld Computers being used with all students or subgroups of 
students? 
O Please provide an example of how you use the Handheld Computers to help 
differentiate instruction. 
d. (If used) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 
oO How were or will these challenges be addressed? 
16. Overall, what are the strengths of the Striving Readers? 
b. In your opinion, what factors do you anticipate will help to facilitate the successful 
implementation (or also described as high fidelity to the model) of Striving Readers during 


the current school year? At the school level? At the classroom level? 


17. Overall, what challenges have you encountered or do you anticipate in the current year to 
implementing the Striving Readers initiative? 


a. How might these challenges be addressed? 
18. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2009 School Visits 
Self Contained Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in identifying best practices 
and gaining a better understanding of how your work can be better supported. [I/another member of the 
evaluation team] had interviewed some of you and/or other classroom teachers about this last fall, and 
today I’d like to follow-up on that conversation. In particular, I’d like to focus on any changes in the 
implementation of the program at your school—overall and for each program component—that have 
occurred since the fall interview. Id also like to touch upon any new successes since last fall, as well as 
the challenges and anticipated developments that you had identified during the fall interview. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 
recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 
specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to you or your school. Is this all right? 


Note to interviewer — How to conduct this interview: 


a) For all topics, probe first for new successes respondents have achieved since the fall interview, 
and request concrete examples of each. 


b) For topics where challenges or anticipated changes are listed that were mentioned last fall, inquire 
about update in the status, including changes in implementation 


1. Please introduce your[self/selves] and tell us how long have you been in this school, and which 
grade level(s) you teach this year. 


We would like to ask you about additional successes you have had in each area since the fall 
interview, as well as the status of any challenges or anticipated changes that you had mentioned last 
fall. Let’s discuss these successes and challenges as they relate to each of the topics that we 
discussed in the fall. 
2. Specific Striving Readers instructional frameworks and materials 

e Instructional Frameworks(for example: whole part whole instruction, differentiating 


instruction, independent reading, small group instruction) 
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e Core ComprehensionStrategies(Summarizing, Visualizing, Questioning, Metacognition, 
Inferring, Predicting, Text structure) 

e Core Comprehension Techniques (for example: Marzano’s vocabulary, PRC2, KWL, Word 
Study/word sorting, Read Aloud/Think Aloud, Reading Response, INSERT notes, ReQuest, List- 
Group-Label, etc.) 


e Striving Readers Instructional Materials (reading response notebooks, vocabulary notebooks, 
text sets, teacher text set guides, etc.) 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


3. Selecting appropriate student reading materials 


e use of fluency snapshots, interest inventories, identifying student reading levels; use of classroom 
libraries for self selected reading 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


4. Use of Grade-Level Team(s) to plan and coordinate instruction and student groupings for 
differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


5. Use of a Literacy Leadership Team to share knowledge and to plan and coordinate instruction 
and student groupings for differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


6. Collaboration with the Literacy Intervention Teacher to provide blended instruction 
(scaffolding, differentiation for all students; addressing the needs of Tier 2 & 3 students) 
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Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


7. Collaboration with the District Coordinator to support implementation of Striving Readers 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


8. Using student data to plan small group activities, monitor success, and select appropriate 
reading materials 


e assessment data, demographic data, behavioral records, etc. 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


9. Comfort and successes/challenges integrating literacy as part of content area instruction, 
including specific Striving Readers instructional frameworks 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


10. Support from key experts for integrating literacy into content area instruction 
Literacy Intervention Teacher 

Lead Literacy Teacher/Literacy Coach 

Striving Readers Coordinator 

Regular ELA teachers 

Other staff member 
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Challenges/anticipated changes from fall interview 
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Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


11. Professional Development provided by Striving Readers to support integration of literacy into 
other content areas 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


12. Other kinds of support to help non-literacy staff integrate literacy into your content area 
instruction? 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


13. Use of Striving Readers texts (i.e. sets of reading materials with differentiated structures and 
levels centered around specific content area themes, designed to improve student literacy in 
other subject area classes) 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


14. Building and using classroom libraries to support instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


15. Using Listening Centers to support differentiated instruction 


Challenges/anticipated changes from fall interview 
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Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


16. UsingMedia Centers to support differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


17. Using Palm Pilots/Handheld Computers to support differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


18. Additional Miscellaneous Challenges/Anticipated Changes from the Fall Interview 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


19. What factors do you anticipate will help to facilitate the successful implementation of Striving 
Readers in the 2009-2010 school year? 


20. What factors might hinder implementation next year? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Fall 2008 School Visits 
ELA Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. Your school was among six schools that were selected for an 
in-depth case study in part because they were identified by the district as being successful in their 
implementation of the Striving Readers initiative. We are interested in learning about the implementation 
of this program in your school, overall and for each program component, identifying best practices, and 
gaining a better understanding of the facilitating conditions and challenges to implementation. Since we 
are requesting a lot of information and I know you have a busy schedule, please be as succinct as you can 
in your answers. You will have an opportunity to elaborate further at the end of the focus group. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 
recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 
specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to yourself or your school. Is this all right? 


1. Please introduce yourselves and tell us how long have you been in this school, and which grade 
level(s) you teach this year. 


2. We would like to know about your use of specificStriving Readers instructional 
frameworks, techniques and strategies. 


a. How comfortable are you with the different frameworks, techniques and strategies 
identified in the Striving Readers grant? When did you start using them? 


b. Which oneshave you found to be the most successful and why? Which ones have 
presented the greatest challenges and why? 


Probe and request examples: 


o Striving Readersinstructional frameworks (for example: whole part whole 


instruction, differentiating instruction, independent reading, small group 
instruction) 


o Core Comprehension Strategies(Summarizing, Visualizing, Questioning, 


Metacognition, Inferring, Predicting, Text structure) 
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o Core Comprehension Techniques (for example: Marzano’s vocabulary, PRC2, 
KWL, Word Study/word sorting, Read Aloud/Think Aloud, Reading Response, 
INSERT notes, PLAN, ReQuest, List-Group-Label, etc.) 


3. We would like to know more about the process that you, teachers and students use to select 
appropriate student reading materials. 


a. Do you use fluency snapshots and interest inventories to guide students’ selection of 
reading materials for their wide reading? Please provide examples. 


b. Are student reading levels used to help select appropriate materials? Please explain and 
provide examples. 


c. To what extent do students know how to select materials for wide reading that are 
appropriate to their abilities? Please explain and provide examples. 


Next, I would like to ask you a few questions about the specifics of when you meet with other 
teachers to plan and coordinate instruction and student groupings for the purpose of providing 
differentiated instruction that is appropriate to each student’s needs. 

4. Does your school have Grade-Level Teams? (Yes (No (IF NO, SKIP TOQS) 


a. Are you all involved in these teams? 


b. What is your role in these teams? What topics are discussed in these meetings? 


5. Does your school have a Literacy Leadership Team? Yes (No (IF NO, SKIP TO Q6) 
a. Are any of you involved in this team? (Yes (No (IF NO, SKIP TO Q6) 


b. What is your role in this team? What topics are discussed in these meetings? 


6. We would like toknow more about your collaboration with the Literacy Intervention Teacher 
and how you plan together to provide blended, differentiated instruction. 


a. In which setting(s) do you meet or collaborate with the LIT? 
O One-on-one meetings: 
Oo Grade-level team meetings: 
oO Literacy leadership team meetings: 


b. Do you collaborate with the LIT in scaffolding instruction (comprehension, vocabulary 
and fluency)? Please provide examples. 


c. Do you plan together with the LIT for blended instruction that differentiates small group 
work? Please provide examples. 
0 Do you collaborate with the LIT to use assessment or other data to fine-tune 
differentiated instruction? Please provide examples. 


d. Please describe other topics you discuss when meeting with the LIT to ensure that Tier 2 


and 3 students receive appropriately targeted, differentiated instruction during classroom 
push-in sessions and AMP after-school lessons. 
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e. Have there been any changes in the strategies, activities and resources that you use during 
these push-in sessions, as compared to last year? 
oO If yes, please describe. 
Oo Why did these changes come about? 


f. What challenges, if any, have you encountered while working with the LIT and/or Tier 2 
and 3 students this year? 
o How were or will these challenges be addressed? 
7. We would like to learn more about your collaboration with the District Coordinator. 
a. Do you meet with him/her? If so, how often do you meet with him/her? 
___ Oncea ___ Several ____ Once ___ Several ___ Daily or 


month or less times a month a week times a week almost daily 


b. In what ways do you collaborate? What topics do you discuss? In what ways does 
he/she support your implementation of Striving Readers? 


c. Is there any additional support they could provide you with? 
8. In what other ways, on your own or in collaboration with the LIT, do you use student data? 
(This might include assessment data as well as other types of student data such as demographic 
and behavioral records.) 


a. Which kinds of data are you using? 


b. In which setting(s) (e.g., individually, as part of grade-level teams, as part of the Literacy 
Leadership Team, other) 


c. For what purposes? Please describe and provide examples. 
Probe & request examples: 

oO Plan for small group activities, 

0 Monitor students’ success in learning techniques for developing comprehension and using 
reading strategies appropriately, 


O Select appropriate materials at students' independent and instructional levels. 


We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 


9. One of the components of Striving Readers is building your classroom libraries. 
a. How do you use the classroom libraries? 


Probe & request examples: 


For independent reading? 
For small group instruction? 


To support content area instruction? 


For read alouds? 
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b. Do you use interest inventories to help students self select reading material and to guide 
your purchases? Please describe the process. 


c. How do you organize books in your classroom library? (Probe: Is there more than one 
criterion used to organize the libraries?) 


d. Have you or your students encountered any challenges in using your classroom libraries 
to support instruction? If so, please describe. 
Oo How have or will these challenges be addressed? 


10. Does your school have Listening Centers (where students can access models of fluency and 
record themselves to assess their own fluency)? [Yes (No (IF NO SKIP TO Q11) 


a. Where are these Listening Centers located? 

Li In the classrooms only 

(| Outside of the classrooms only (e.g., computer lab) 
oO How accessible are they? 

Both 


ai 


b. Do you use the Listening Centers? [Yes [No 
c. (Ifnot used) Why not? 


d. (fused) How are you using them? 
fe) For which type of activities? 
fe) Are you using the Listening Centers with all students or subgroups of these 
students? 
fe) Please provide an example of how you use the Listening Centers to help 
differentiate instruction. 


e. (If used) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 
Oo How were or will these challenges be addressed? 


11. Do you have Media Centers (3 computers and 1 printer) in your classrooms)? (|Yes_ (INo (IF 
NO SKIP TO Q12) 


a. Do you use the Media Centers? (Yes UINo 

b. (If not used) Why not? 

c. (If used) How are you using them? 
O For which type of activities? 
O Are you using the Media Centers with all students or subgroups of these students? 
O Please provide an example of how you use the Media Centers to help differentiate 


instruction. 


d. (Ifused) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 
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oO How were or will these challenges be addressed? 
12. Do you and/or students in your school have access to Palm Pilots/Handheld Computers? 
(Yes (| No? Ifno, why not? (THEN SKIP TO Q13) 
a. Do you and/or your students use them in your classroom? [1Yes LINo 
b. (If not used either by Teachers or by Students) Why not? 
c. (fused), how are they being used? 
oO For which type of activities do you use the Handheld Computers? 
o Are the Handheld Computers being used with all students or subgroups of 
students? 


O Please provide an example of how you use the Handheld Computers to help 
differentiate instruction. 


d. (if used) Have you or your students encountered any challenges when using this technology? 
If so, please describe. 
oO How were or will these challenges be addressed? 
13. Overall, what are the strengths of the Striving Readers? 
a. In your opinion, what factors do you anticipate will help to facilitate the successful 
implementation (or also described as high fidelity to the model) of Striving Readers during 


the current school year? At the school level? At the classroom level? 


14. Overall, what challenges have you encountered or do you anticipate in the current year to 
implementing the Striving Readers initiative? 


a. How might these challenges be addressed? 
15. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2009 School Visits 
ELA Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in identifying best practices 
and gaining a better understanding of how your work can be better supported. [I/another member of the 
evaluation team] had interviewed some of you and/or other ELA teachers about this last fall, and today 
I'd like to follow-up on that conversation. In particular, I’d like to focus on any changes in the 
implementation of the program at your school—overall and for each program component—that have 
occurred since the fall interview. I’d also like to touch upon any new successes since last fall, as well as 
the challenges and anticipated developments that you had identified during the fall interview. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 
recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 
specific comments that you would like to keep confidential, let us know and we will make sure they 
cannot be tied to you or your school. Is this all right? 


Note to interviewer — How to conduct this interview: 


a) For all topics, probe first for new successes respondents have achieved since the fall interview, 
and request concrete examples of each. 


b) For topics where challenges or anticipated changes are listed that were mentioned last fall, inquire 
about update in the status, including changes in implementation 


1. Please introduce your[self/selves] and tell us how long have you been in this school, and which 
grade level(s) you teach this year. 


We would like to ask you about additional successes you have had in each area since the fall 
interview, as well as the status of any challenges or anticipated changes that you had mentioned last 
fall. Let’s discuss these successes and challenges as they relate to each of the topics that we 
discussed in the fall. 


2. Specific Striving Readers instructional frameworks and materials 
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e Instructional Frameworks(for example: whole part whole instruction, differentiating 
instruction, independent reading, small group instruction) 

e Core Comprehension Strategies(Summarizing, Visualizing, Questioning, Metacognition, 
Inferring, Predicting, Text structure) 

e Core Comprehension Techniques (for example: Marzano’s vocabulary, PRC2, KWL, Word 
Study/word sorting, Read Aloud/Think Aloud, Reading Response, INSERT notes, ReQuest, 
List-Group-Label, etc.) 

e Striving Readers Instructional Materials (reading response notebooks, vocabulary notebooks, 


text sets, teacher text set guides, etc.) 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


3. Selecting appropriate student reading materials. 


e use of fluency snapshots, interest inventories, identifying student reading levels; use of classroom 
libraries for self selected reading 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


4. Use of Grade-Level Team(s)to plan and coordinate instruction and student groupings for 
differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


5. Use of Literacy Leadership Teamto share knowledge and to plan and coordinate instruction 
and student groupings for differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 
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6. Collaboration with the Literacy Intervention Teacher to provide blended instruction 
(scaffolding, differentiation for all students; addressing the needs of Tier 2 3 students) 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


7. Collaboration with the District Coordinator to support implementation of Striving Readers 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


8. Using student data to plan small group activities, monitor success, and select appropriate 
reading materials 


e assessment data, demographic data, behavioral records, etc. 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


9. Using your classroom libraries to support instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


10. UsingListening Centersto support differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 
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11. Using Media Centers (3 computers and 1 printer)to support differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


12. Using Palm Pilots/Handheld Computers to support differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


13. Additional Miscellaneous Challenges/Anticipated Changes from the Fall Interview 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


14. What factors do you anticipate will help to facilitate the successful implementation of Striving 


Readers in the 2009-2010 school year? 


15. What factors might hinder implementation next year? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Fall 2008 School Visits 
Non-ELA Content Area Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. Your school was among six schools that were selected for an 
in-depth case study in part because they were identified by the district as being successful in their 
implementation of the Striving Readers initiative. We are interested in learning about the implementation 
of this program in your school, overall and for each program component, identifying best practices, and 
gaining a better understanding of the facilitating conditions and challenges to implementation. Since we 
are requesting a lot of information and I know you have a busy schedule, please be as succinct as you can 
in your answers. You will have an opportunity to elaborate further at the end of the focus group. 


This focus group will take about 60 minutes. I would like to tape this focus group to be sure I have 

recorded it accurately. The recordings will not be shared with anyone outside of Metis. If there are any 

specific comments that you would like to keep confidential, let us know and we will make sure they 

cannot be tied to yourself or your school. Is this all right? 

1. [Interviewer: please go around the room and ask participants to introduce themselves] Please 
introduce yourselves and tell us how long you have been a teacher in this school, and which 
subject area(s) and what grade level(s) you teach this year. 


We would like to know more about the types of support that you have received this year to help you 
integrate literacy into other content area instruction. 


2. Have you met with any of the following literacy experts? How often? What did you discuss with 
them? 


a. Literacy Intervention Teacher? 

b. Lead Literacy Teacher/Literacy Coach? 
c. Striving Readers Coordinator? 

d. Regular ELA teachers? 


e. Other staff member(s)? Please specify: 


3. Have youever participated in any professional development conducted by the Striving Readers 
program related to integrating literacy into other content area instruction? LYes LINo 
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a. If yes, when/what year did you start participating in this type of trainings? 


b. Which trainings did you participate in? (Probe and request specific examples: site-based 
professional development, Summer Institute and follow-up Institutes, Saturday Seminars) 


c. Which trainings were most useful and why? Which trainings were least useful and why? 
d. What areas or topics would you like to receive additional support or training in? 


4. What other kinds of support are availableto help non-literacy staff integrate literacy into your 
content area instruction? 


5. What other kinds of support would you need to help you integrate literacy into your content 
area instruction? 


The next few questions are about the extent and relative success of your efforts tointegrate literacy 
as part of your content area instruction. 


6. When, if at all,did you start integrating literacy into your content area instruction? (Probe for: 
this year, last year, before Striving Readers began?) 


7. Have you been able to use any of the specific frameworks, techniques and strategies identified 
in the Striving Readers grant? 


a. Ifnot, how have you approached integrating literacy in your classroom? 


b. Ifyou have used specific Striving Readers frameworks, techniques and strategies, please 
provide examples. 


Probe and request examples: 


o Striving ReadersInstructional Frameworks(for example: whole part whole 


instruction, differentiating instruction, independent reading, small group 
instruction) 


o Core Comprehension Strategies(Summarizing, Visualizing, Questioning, 


Metacognition, Inferring, Predicting, Text structure) 


o Core Comprehension Techniques (for example: Marzano’s vocabulary, PRC2, 
KWL, Word Study/word sorting, Read Aloud/Think Aloud, Reading Response, 
INSERT notes, PLAN, ReQuest, List-Group-Label, etc.) 


8. Does your school have school-wide text sets (i.e., sets of reading materials of different structures 
and levels, centered around specific content area themes, designed to improve student literacy 
in other subject area classes)? “Yes (CINo (IF NO SKIP TO Q9) 

a. Are the school-wide text sets being used in the content area classrooms? 
Oo Social Studies 0 Not Used—why? 5 Used—how? o Don’t Know 
oO Science o Not Used—why? o Used—how? o Don’t Know 
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9. One of the components of Striving Readers is building your classroom libraries. 


a. How do you use the classroom libraries? To what extent do they support your efforts to 
integrate literacy into other content area instruction? 


Probe & request examples: 
For content area? 


For independent reading? 


For small group instruction? 


For read alouds? 


b. Do you use interest inventories to help students self select reading material and to guide your 
purchases? 


c. How do you organize books in your classroom library? (Probe: Is there more than one 
criterion used to organize the libraries?) 


16. To what extent have you used technology to support your efforts to integrate literacy into other 
content area instruction? 


a. Please describe and provide examples. [Probe: media centers (classroom computers and 
printers), listening centers, hand held computers] 


b. Have there been any changes in the use of technology as compared to last year? 
c. Have you and/or the students encountered any challenges when using technology? If so, 
please describe. 


Oo How were or will these challenges be addressed? 


We would also like to know more about the factors facilitating or hindering the implementation of 
Striving Readers as it relates to the integration of literacy instruction into the other content areas. 


16. In your opinion, what factors do you anticipate will help to facilitate the successful integration 
of literacy instruction into the other content areas during the current school year? At the school 


level? At the classroom level? 


17. Overall, what challenges have you encountered or do you anticipate in the current year related 
to integrating literacy into the content areas? 


a. How might these challenges be addressed? 
18. Is there anything else you would like to add? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2009 School Visits 
Non-ELA Content Area Teachers Focus Group Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in identifying best practices 
and gaining a better understanding of how your work can be better supported. [I/another member of the 
evaluation team] had interviewed some of you and/or other subject area teachers about this last fall, and 
today I’d like to follow-up on that conversation. In particular, I’d like to focus on any changes in the 
implementation of the program at your school—overall and for each program component—that have 
occurred since the fall interview. I’d also like to touch upon any new successes since last fall, as well as 
the challenges and anticipated developments that you had identified during the fall interview. 


This focus group will take about 60 minutes. I would like to tape this interview to be sure I have recorded 
it accurately. The recordings will not be shared with anyone outside of Metis. If there are any specific 
comments that you would like to keep confidential, let us know and we will make sure they cannot be tied 
to you or your school. Is this all right? 


Note to interviewer — How to conduct this interview: 


a) For all topics, probe first for new successes respondents have achieved since the fall interview, 
and request concrete examples of each. 


b) For topics where challenges or anticipated changes are listed that were mentioned last fall, inquire 
about update in the status, including changes in implementation 


1. Please introduce your[self/selves] and tell us how long you have been a teacher in this school, 
and which subject area(s) and what grade level(s) you teach this year. 


We would like to ask you about additional successes you have had in each area since the fall 
interview, as well as the status of any challenges or anticipated changes that you had mentioned last 
fall. Let’s discuss these successes and challenges as they relate to each of the topics that we 
discussed in the fall. 


2. Support from key experts for integrating literacy into content area instruction 
a. Literacy Intervention Teacher 
b. Lead Literacy Teacher/Literacy Coach 
c. Striving Readers Coordinator 
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d. Regular ELA teachers 
e. Other staff member 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


3. Professional Development provided by Striving Readers to support integration of literacy into 
other content areas 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


4. Other kinds of support to help non-literacy staff integrate literacy into your content area 
instruction? 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


5. Integrating literacy as part of content area instruction, including specific Striving Readers 
instructional frameworks and materials 


O Instructional Frameworks(for example: whole part whole instruction, 
differentiating instruction, independent reading, small group instruction) 

O Core Comprehension Strategies(Summarizing, Visualizing, Questioning, 
Metacognition, Inferring, Predicting, Text structure) 

O Core Comprehension Techniques (for example: Marzano’s vocabulary, PRC2, 


KWL, Word Study/word sorting, Read Aloud/Think Aloud, Reading Response, INSERT 
notes, ReQuest, List-Group-Label, etc.) 


O Striving Readers Instructional Materials (reading response notebooks, 


vocabulary notebooks, text sets, teacher text set guides, etc.) 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 
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6. Use of school-wide texts (i.e. sets of reading materials with differentiated structures and levels 
centered around specific content area themes, designed to improve student literacy in other 
subject area classes) 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


7. Building and using classroom libraries to support literacy instruction in your content area 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


8. Use of technology to support integration of literacy into content area instruction 
f. Listening Centers 


g. Media Centers (3 computers and 1 printer) 
h. Palm Pilots/Handheld Computers 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


9. Additional Miscellaneous Challenges/Anticipated Changes from the Fall Interview 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


10. What factors do you anticipate will help to facilitate the successful integration of literacy into 
the content areas in the 2009-2010 school year? 


11. What factors might hinder integration of literacy into the content areas next year? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Fall 2008 School Visits 
Literacy Intervention Teacher (LIT) Interview Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. Your school was among six schools that were selected for an 
in-depth case study in part because they were identified by the district as being successful in their 
implementation of the Striving Readers initiative. We are interested in learning about the implementation 
of this program in your school, overall and for each program component, identifying best practices, and 
gaining a better understanding of the facilitating conditions and challenges to implementation. Since we 
are requesting a lot of information and we have a limited period of time, please be as succinct as you can 
in your answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. The recordings will not be shared with anyone outside of Metis; however, your comments 
may be used in reporting findings from this case study. If there are any specific comments that you would 
like to keep confidential, let us know and we will make sure they cannot be tied to yourself or your 
school. Is this all right? 


1. How long have you been an LIT? If you were in this role last year, has your role as the Literacy 
Intervention Teacher (LIT) in the Striving Readers Initiative changed from last year? 
a. Ifso, how? 
2. We would like to learn more about your collaboration with the District Coordinator. 
a. How often do you meet with him/her? 
___ Oncea ___ Several ____ Once ____ Several ____ Daily or 


month or less times a month a week times a week almost daily 


b. What topics do you discuss? In what ways does he/she support your implementation of 
Striving Readers? 


c. Is there any additional support they could provide you with? 


Next, I would like to ask you a few questions about the specifics of when you meet with classroom 
teachers to plan, prioritize and coordinate instruction, responsibilities, and student groupings. 


3. Do you have one-on-one meetings with the teachers outside of instruction time? 
Yes LINo (IF NO SKIP TO Q4) 


a. When do you have these meetings? 
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__Oncea ___ Several ____ Once ___ Several ____ Daily or 
month or less times a month a week times a week almost daily 


b. Please describe how these meetings are used to ensure that Tier 2 and 3 students receive 
appropriately differentiated instruction during the classroom targeted intervention and/or 
after school classes. 


c. What topics are discussed in these meetings? 


d. Do you discuss assessment data other types of student data (such as demographic data or 
behavioral records) in these meetings? If so, for what purposes 


Does your school have Grade-Level Teams? (Yes (No (IF NO, SKIP TO Q5) 


a. Are you involved in these teams? (Yes (No (IF NO, SKIP TO Q5) 


b. Please describe how these meetings are used to ensure that Tier 2 and 3 students receive 
appropriately targeted, differentiated instruction during the classroom targeted 
intervention and/or after school classes. [If applicable]: How does this function differ 
from that of the one-on-one meetings? 


c. Do these teams use assessment data, orother types of student data (such as demographic 
data or behavioral records)? If so, for what purposes? [If applicable]: How does this 
differ from the use of data during one-on-one meetings? 


Does your school have a Literacy Team? [Yes (INo (IF NO, SKIP TO Q6) 


a. Are you involved in this team? “Yes (CINo (IF NO, SKIP TO Q6) 


b. Please describe how these meetings are used to ensure that Tier 2 and 3 students receive 
appropriately targeted, differentiated instruction during the classroom targeted intervention 
and/or after school classes.[If applicable]: How does this function differ from those of the 
one-on-one and Grade-Level Team meetings? 


c. Do these teams use assessment data, orother types of student data (such as demographic data 
or behavioral records)? If so, for what purposes? [If applicable]: How does this differ from 
the use of data during one-on-one and Grade-Level Team meetings? 

Have you encountered any challenges in collaborating with classroom teachers one-on-one or as 
part of the team meetings? 


a. Ifso, how were or will these challenges be addressed? 


I would like to know more about the targeted intervention and the work that you do with Tier 2 
and Tier 3 students during the regular school day. 


a. Which grades do you work with? How many classes? For how long (e.g., length of targeted 
instruction, number of periods per week per class)? 


b. On average, how many kids are there in the targeted group per class? 
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8. 


How do you determine the needs of your students that might impact on their literacy 
development? 


How do you develop appropriate instruction for them? 


What types of strategies, activities and resources do you use when you meet with your tier 2 
& 3 students? 
oO What types of student grouping do you use? 
O What strategies do you use to differentiate instruction for students of different 
ability levels within this group? 


Have you encountered any challenges when working in the language arts classroom with Tier 
2 and 3 students and/or their teachers this year? If so, please describe. 
Oo How were or will these challenges be addressed? 


Have there been any changes in the strategies, activities and resources that you use during 
these targeted intervention sessions, as compared to last year? 

oO If yes, please describe. 

Oo Why did these changes come about? 


I would like to know more about the intensive intervention and the work that you do with Tier 
3 students during the after school program. 


a. 


b. 


What is your role in the after school component of Striving Readers? 


Please describe the structure and content of the after school Achieving Maximum Potential 
(AMP) programming for struggling readers. 


Are you using the AMP intervention software during the afterschool program? 1Yes [No 
oO (Ifnot used) Why not? 
oO (If used) How is it being used? 


1. For which type of activities? 
2. Are you using it with all students or subgroups of students? 


oO (Ifused) Have you, students, and/or teachers encountered any challenges when 
using this technology? If so, please describe. 


"How were or will these challenges be addressed? 


oO (Ifused) In what ways, if any, has this technology improved instruction and student 
learning in language arts? 


What successes has the school had with the after school component of Striving Readers? 
What challenges has the school encountered with the after school component of Striving 
Readers? 


oO How were or will these challenges be addressed? 


Have there been any changes in the strategies, activities and resources that you use during the 
afterschool program, as compared to last year? 
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oO If yes, please describe. 
Oo Why did these changes come about? 


We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 


9. Do your teachers utilize the professional library in the school? If so, how? 


10. In what ways are the classroom libraries being used by the teachers? For what types of 
activities? 


a. What is your role, if any, in helping teachers use their classroom libraries? 


11. Does your school have Listening Centers (where students can access models of fluency and 
record themselves to assess their own fluency)? [Yes (No (IF NO SKIP TO Q12) 


a. 


Do you use the Listening Centers to support your role in providing differentiated instruction 
to struggling readers? Yes LINo 


(if not used) Why not? 


(If used) How are you using them? 
fe) For which type of activities? 
oO Are you using the Listening Centers with all struggling students or subgroups of 
these students? 
fe) Please provide an example of how you use the Listening Centers to help 
differentiate instruction. 


(If used) Have you, students, and/or teachers encountered any challenges when using this 
technology? If so, please describe. 
oO How were or will these challenges be addressed? 


12. Does your school have Media Centers (3 computers and 1 printer in the classroom)? || Yes 
LINo (IF NO SKIP TO Q13) 


a. 


Do you use the Media Centers to support your role in providing differentiated instruction to 
struggling readers? (Yes [INo 


(if not used) Why not? 


(If used) How are you using them? 
0) For which type of activities? 
O Are you using the Media Centers with all struggling students or subgroups of these 
students? 
O __— Please provide an example of how you use the Media Centers to help differentiate 
instruction. 


Have you, students, and/or teachers encountered any challenges when using this technology? 
If so, please describe. 
oO How were or will these challenges be addressed? 
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13. Do you and/or students in your school have access to Palm Pilots/Handheld Computers? 
‘Yes (1 No? Ifno, why not? (THEN SKIP TO Q14) 


a. Do you use the Handheld Computers to support your role in providing differentiated 
instruction to struggling readers? [Yes [INo 


b. Do your students use the Palm Pilots/Handheld Computers? Yes ([INo 
c. (Ifnot used either by LIT or by Students) Why not? 


d. (If used) Are they being used during the afterschool program? (Yes [INo 
0 If yes, how are they being used? 
1. For which type of activities do you use the Handheld Computers? 
2. Are the Handheld Computers being used with all struggling readers or 
subgroups of students? 
3. Please provide an example of how you use the Handheld Computers to help 
differentiate instruction. 


e. (If used) Are they being used as part of the targeted intervention during the regular school 
day? “Yes (CNo 


0 If yes, how are they being used? 
"For which type of activities do you use the Handheld Computers? 
« Are the Handheld Computers being used with all struggling readers or 
subgroups of students? 
" Please provide an example of how you use the Handheld Computers to help 
differentiate instruction. 


f. Have you, students, and/or teachers encountered any challenges when using this technology? 
If so, please describe. 
Oo How were or will these challenges be addressed? 
14. Overall, what are the strengths of your school’s literacy curriculum? 
b. In your opinion, what factors do you anticipate will help to facilitate the implementation of 
Striving Readers as defined by the model during the current school year? At the school level? 


At the classroom level? 


15. Overall, what challenges have you encountered or do you anticipate in the coming year to 
implementing the Striving Readers initiative? 


a. How might these challenges be addressed? 
16. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2009 School Visits 
Literacy Intervention Teacher (LIT) Interview Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in identifying best practices 
and gaining a better understanding of how your work can be better supported. [I/another member of the 
evaluation team] had interviewed you about this last fall, and today I’d like to follow-up on that 
conversation. In particular, I’d like to focus on any changes in the implementation of the program at your 
school—overall and for each program component—that have occurred since the fall interview. I’d also 
like to touch upon any new successes since last fall, as well as the challenges and anticipated 
developments that you had identified during the fall interview. 


This interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. The recordings will not be shared with anyone outside of Metis. If there are any specific 
comments that you would like to keep confidential, let us know and we will make sure they cannot be tied 
to you or your school. Is this all right? 


Note to interviewer — How to conduct this interview: 


a) For all topics, probe first for new successes respondent has achieved since the fall interview, and 
request concrete examples of each. 


b) For topics where challenges or anticipated changes are listed that were mentioned last fall, inquire 
about update in the status, including changes in implementation 


We would like to ask you about additional successes you have had in each area since the fall 
interview, as well as the status of any challenges or anticipated changes that you had mentioned last 
fall. Let’s discuss these successes and challenges as they relate to each of the topics that we 
discussed in the fall. 


1. Collaboration with the District Coordinator to support the implementation of Striving Readers 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 
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2. One-on-one collaboration with teachers outside of instruction time to plan how to differentiate 
instruction during classroom targeted interventions or to discuss assessment/student data 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


3. Use of Grade-Level Team(s) to plan and coordinate instruction and student groupings for 
differentiated instruction or to discuss assessment/student data 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


4. Use of Literacy Leadership Team to share knowledge and to plan and coordinate instruction 
and student groupings for differentiated instruction or to discuss assessment/student data 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


5. Collaborating with classroom teachers one-on-one or as part of the team meetings 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


6. Targeted Intervention and work done with Tier 2 and Tier 3 students during regular school day 
— identifying and meeting the needs of struggling readers 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
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Probe: have any new challenges arisen since the fall interview. 


7. The intensive intervention and your work with Tier 3 students during the after school program 
(AMP) 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


8. Your teachers’ use of the school’s professional library 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


9. Your teacher’s use of classroom libraries to support instruction? 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


10. Using Listening Centers to support differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


11. Using Media Centers (3 computers and 1 printer in the classroom) to support differentiated 
instruction 


Challenges/anticipated changes from fall interview 
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Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


12. Using Palm Pilots/Handheld Computers to support differentiated instruction? 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


13. Additional Miscellaneous Challenges/Anticipated Changes from the Fall Interview 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


14. What factors do you anticipate will help to facilitate the successful implementation of Striving 
Readers in the 2009-2010 school year? 


15. What factors might hinder implementation next year? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Fall 2008 School Visits 
Principal Interview Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. Your school was among six schools that were selected for an 
in-depth case study in part because they were identified by the district as being successful in their 
implementation of the Striving Readers initiative. We are interested in learning about the implementation 
of this program in your school, overall and for each program component, identifying best practices, and 
gaining a better understanding of the facilitating conditions and challenges to implementation. Since we 
are requesting a lot of information and I know you have a busy schedule, please be as succinct as you can 
in your answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. The recordings will not be shared with anyone outside of Metis; however, your comments 
may be used in reporting findings from this case study. If there are any specific comments that you would 
like to keep confidential, let us know and we will make sure they cannot be tied to you or your school. Is 
this all right? 


1. How long have you been the principal in this building? [If more than one year] Has your role 
in the Striving Readers Initiative changed from last year? 
a. Ifso, how? 


2. Does your school have a Literacy Team? [Yes [INo (IF NO, SKIP TO Q3) 


a. Which of your staff are members of the Literacy Team? 


1] Principal (| Grade level teacher(s) (| Literacy Intervention Teacher 
(| Librarian(s) (| Lead Literacy Teacher [|] ELL/ESL Teacher(s) 
(| Special education teacher(s) (| Other: 


b. How often does the Literacy Team meet? 


_| Has not met (| Less than once per month (| Once per month 


(| Weekly 


(| Biweekly (| Several times a week or more 


c. What role does the Literacy Team play at your school? 
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d. What role do you play in the Literacy Team? 
e. How does the team address the needs of struggling readers? 
3. Does your school have Grade-Level Teams? Yes UNo (IF NO, SKIP TO Q4) 
a. Which of your staff are members of the Grade-Level Teams? 
b. How often do these teams meet? 


_| Have not met (| Less than once per month (| Once per month 


(| Biweekly (| Weekly (| Several times a week or more 


c. What role do these teams play at your school? 


d. How do these teams address the needs of struggling readers? 


We would like to learn more about the use of assessment data and how that impacts instruction. 


4. In what ways, if any, is your school using assessment data beyond mandated reporting to the 
district and state? 


a. Please describe and provide examples, including distinctions between how different 
teams or individuals use the data. 
oO Student placement in specific groups, programs, etc. 
o Differentiate instruction 
oO Planning professional development 
o Other 


b. By whom? 
oO Principal 
oO Literacy Leadership Team 
Oo Grade-Level Teams 
Oo Other 
5. Are any other types of data (e.g., demographic, behavioral) being used? 
a. By whom? 


b. For what purposes? Please describe and provide examples. 


6. The AMP after school program is part of Striving Readers. We want to know more about your 
school’s use of this program. 


a. Do you feel that the AMP program is appropriate to the reading levels of the students who are 
currently participating? Are there any students in AMP who you feel should not be there? 
(Reading levels too high/too low?) Are there students who are not in AMP who should be? 


b. Is the LIT or other after-school teachers using the AMP activities and materials? Are they 
using any supplemental materials or strategies? 
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c. Are students using the AMP software? Yes LINo 


o Ifnot, why not? 


d. What successes has the school had with implementing the AMP program strategies, including 
the use of the AMP software? 


e. What challenges has the school encountered with the AMP program and software? 


We would like to know more about your use of Striving Readers classroom-based intervention 
materials and technology. 


7. To what extent has the use of technology been integrated into literacy instruction? 


a. 


Please describe and provide examples. [Probe: media centers (classroom computers and 
printers), listening centers, hand held computers] 


Have there been any changes in the use of technology as compared to last year? 
Have the students and/or teachers encountered any challenges when using technology? If 


so, please describe. 
oO How were or will these challenges be addressed? 


In what ways, if any, has this technology improved instruction and student learning in 
language arts? 


8. Now I would like to ask about your school’s efforts, if any, to integrate literacy into the content 


areas. 


Cc. 


Please describe your school’s efforts, if any, to integrate literacy into the content areas. 


Does your school have school-wide text sets (i.e., sets of reading materials of different 
structures and levels, centered around specific content area themes, designed to improve 
student literacy in other subject area classes)? Yes LINo 


oO Ifyes, are the school-wide text sets being used in the content area classrooms? 


Social Studies o Not Used—why? o Used—how? 0 Don’t Know 


Science 0 Not Used—why? 0 Used—how? o Don’t Know 


Mathematics o Not Used—why? o Used—how? o Don’t Know 


Do non-literacy staff participate in professional development for the Striving Readers 
project? (Includes: Bilingual, SPED, Math, Science, Social Studies teachers) (1 Yes No 


oO If yes, who has received professional development? Please specify staff positions. 
oO What topics were covered? 
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d. What other kinds of support are there to help non-literacy staff integrate literacy into their 
content areas? 


e. What are some of the challenges that these teachers are facing? 
Oo How might these challenges be addressed? 


Now we would like to know more about different types of support that your school has received for 
the implementation of Striving Readers. 


9. Please describe your school’s partnership with National-LouisUniversity and Donna Ogle. 
a. In what ways has her support been helpful to the implementation of Striving Readers? 
b. Is there any additional support she could provide you with? 


10. What type of support has the District Coordinator provided to you and your school? Please 
describe. 


a. To what extent has his or her support been helpful for the implementation of Striving 
Readers? Please explain. 


b. Is there any additional support they could provide you with? 


11. Has your role as instructional leader changed as a result of your participation in Striving 
Readers? If so, how? 


12. Overall, what are the strengths of your school’s literacy curriculum? 
a. In your opinion, what factors do you anticipate will help to facilitate the implementation 
of Striving Readers as defined by the model during the current school year? At the school 


level? At the classroom level? 


13. Overall, what challenges have you encountered or do you anticipate in the coming year to 
implementing the Striving Readers initiative? 


a. How might these challenges be addressed? 
14. Is there anything else you would like to add regarding the literacy activities in your school? 


Thank you for your time today. 
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CPS Striving Readers 
Case Study — Spring 2009 School Visits 
Principal Interview Protocol 


Interviewee Name: Date: 


Interviewee Title: 


School: Start Time: End Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public Schools 


Striving Readers external evaluation team. As you know, your school was among six schools that were 
selected for an in-depth case study in part because they were identified by the district as being successful 
in their implementation of the Striving Readers initiative. We are interested in identifying best practices 
and gaining a better understanding of how your work can be better supported. [I/another member of the 
evaluation team] had interviewed you about this last fall, and today I’d like to follow-up on that 
conversation. In particular, I’d like to focus on any changes in the implementation of the program at your 
school—overall and for each program component—that have occurred since the fall interview. I’d also 
like to touch upon any new successes since last fall, as well as the challenges and anticipated 
developments that you had identified during the fall interview. 


This interview will take about 60 minutes. I would like to tape this interview to be sure I have recorded it 
accurately. The recordings will not be shared with anyone outside of Metis. If there are any specific 
comments that you would like to keep confidential, let us know and we will make sure they cannot be tied 
to you or your school. Is this all right? 


Note to interviewer — How to conduct this interview: 


a) For all topics, probe first for new successes respondent has achieved since the fall interview, and 
request concrete examples of each. 


b) For topics where challenges or anticipated changes are listed that were mentioned last fall, inquire 
about update in the status, including changes in implementation 


We would like to ask you about additional successes you have had in each area since the fall 
interview, as well as the status of any challenges or anticipated changes that you had mentioned last 
fall. Let’s discuss these successes and challenges as they relate to each of the topics that we 
discussed in the fall. 


1. Use of a Literacy Team to share knowledge and to plan and coordinate instruction and student 
groupings for differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
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Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


2. Use of Grade-Level Team(s) to plan and coordinate instruction and student groupings for 


differentiated instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


3. Use of assessment data to inform literacy instruction or plan professional development 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


4. Use of other student data (e.g., demographic, behavioral) to inform literacy instruction 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


5. Use of the AMP after school program to support literacy instruction at your school 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


6. Use of technology to support literacy instruction 
e Listening Centers 


e Media Centers (3 computers and 1 printer) 
e Palm Pilots/Handheld Computers 


Challenges/anticipated changes from fall interview 
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Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


7. Integration of literacy instruction into content areas 
e Use of school-wide text sets 
e professional development for non-literacy staff 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


8. Support provided by National-LouisUniversity and Donna Ogle. 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


9. Support provided by Striving Readers District Coordinator 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


10. Effect of Striving Readers on your role as instructional leader 


Challenges/anticipated changes from fall interview 


Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


11. Additional Miscellaneous Challenges/Anticipated Changes from the Fall Interview 


Challenges/anticipated changes from fall interview 
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Probe: new successes since the fall interview, including concrete examples 
Discuss update in status of above challenges, including changes in implementation 
Probe: have any new challenges arisen since the fall interview. 


12. What factors do you anticipate will help to facilitate the successful integration of literacy into 
the content areas in the 2009-2010 school year? 


13. What factors might hinder implementation next year? 


Thank you for your time today. 
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Appendix A-3: Year 2 Measures 


This Appendix includes copies of the following surveys, instruments, and protocols used in the 
evaluation of the Chicago Striving Readers program: 


e Spring 2008 Literacy Improvement Survey for Teachers — Control Schools 
e Spring 2008 Literacy Improvement Survey for Teachers — Treatment Schools 
e Observation Forms: 


Pre-Observation Literacy Environment Checklist 
Checklist of Observation Codes 

Observation Code Definitions 

Literacy Instruction Observation Field Notes Form 
Observation Summary Form 

Pre-Observation Form for Teachers 


e Interview Protocols: 


Literacy Intervention Teacher Interview Protocol — Treatment Schools 
Principal Interview Protocol — Control Schools 

Principal Interview Protocol — Treatment Schools 

Lead Literacy Teacher Interview Protocol — Control Schools 

Lead Literacy Teacher Interview Protocol — Treatment Schools 
Librarian Interview Protocol — Control Schools 

Librarian Interview Protocol — Treatment Schools 

Technology Coordinator Interview Protocol — Control Schools 
Technology Coordinator Interview Protocol — Treatment Schools 
Librarian/Technology Coordinator Interview Protocol — Control Schools 
Librarian/Technology Coordinator Interview Protocol — Treatment Schools 
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Chicago Public Schools (CPS) Striving Readers 
Spring 2008 Literacy Improvement Survey for Teachers — Control Schools 


The following is a survey designed to gather your feedback on the essential components of your school’s literacy program. It will take 
you approximately 30 minutes to complete and results will be reported in the aggregate only. We will not use your name or identify 
individual respondents. Your feedback is extremely valuable to the success of this program. If you have questions about this survey, 
please contact Rebecca Swann at rswann@metisassoc.com or 212-425-8833. 


COMPREHENSIVE INSTRUCTION 


1. How often do you use the following practices to help struggling readers develop better reading strategies and skills? 


Less than : , , 
z ‘ 1-3 timesa | 1-3 timesa | 4-5 times a : 
Practices and Techniques Never once a Don’t know 
winath month week week 


Employing direct vocabulary instruction. 


Differentiating instruction. 


sing guided reading. 


sing partner reading for fluency. 


sing partner reading for comprehension. 


U 
U 
U 
U 


sing partner reading for vocabulary development. 


Scaffolding. 


Using whole-part-whole classroom instruction model 


Establishing the purpose for reading. 


Making connections to background knowledge. 


Understanding the arrangement of text. 


Making connections between texts. 


Monitoring comprehension through questioning. 


Synthesizing information within text or across texts. 
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2. How often do you use the following techniques to help struggling readers develop better reading strategies and skills? 


Techniques 


Never/ Not 
Familiar 


Less than 
once a month 


1-3 times a 
month 


1-3 times a 
week 


4-5 times a 
week 


Everybody Reads To (ERT) 


Exclusion Brainstorming 


List-Group-Label 


Predict-Locate-Add-Note (PLAN) 


ReQuest 


Interactive Notation System for Effective Reading and 
Thinking (INSERT) 


Reciprocal teaching 


ABC Graffiti 


Guided Reading and Summarizing Procedure (GRASP) 


STRUGGLING READERS: EXTENDED DAY (AFTERSCHOOL) INTERVENTION 


3. Does your school currently offer afterschool programming specifically targeting struggling readers? 


Oo Yes 
Oo No(f no, skip to Question 6.) 


4. How many of your current students are involved in the afterschool program? 


O None (If none, skip to Question 6.) 
1 to3 

4to6 

7to9 

10 or more 


o0O0°0 
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5. Overall, how effective has the afterschool component been in improving the literacy abilities of struggling readers? 
O Notat all effective 

Minimally effective 

Somewhat effective 

Effective 

Very effective 

Don’t know 


oOo00°0 


PURPOSEFUL ASSESSMENT 


6. Does your school have a lead literacy teacher or literacy coach? 
Oo Yes 
oO No (If no, skip to Question 8.) 


7. To what extent do you work with your lead literacy teacher/literacy coach to use assessment data for instructional planning? 
Oo Notat all 
Oo Toasmall extent 
Oo Toamoderate extent 
o Toa large extent 
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8. Indicate how you use the data from the following assessments. (Check all that apply.) 


Progress Assess 


Assessments Not Using | Screening | Diagnostic | Benchmarking Monitoring | Outcomes 


Learning FirstClass Views 


Learning First ClassLinks 


mClass Running Records 


IllinoisStandards Achievement Test 


Basic Reading Inventory (BRI) 


Informal assessments 


Other (please specify): 


Other (please specify): 


Other (please specify): 


DATA-DRIVEN INSTRUCTION 


9. Please indicate the extent to which you use student assessment data for each of the following purposes. 


To a Moderate To a Large 


Use of Data Not at All To Some extent 
extent extent 


Placing students in intervention programs. 


Differentiating instruction. 


Identifying skills that need to be retaught. 


Monitoring student reading progress. 


Creating instructional groups (in-class). 
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GRADE-LEVEL TEAMS 


10. Do you currently have grade-level teams at your school? 


11. Overall, rate the grade-level team’s ability to use classroom assessment data in the following ways. 


Oo Yes 
O No (If no, skip to Question 13.) 


Use of Data Poor 


Fair 


Good 


Excellent 


Not Sure 


Address the needs of struggling readers. 


Formalize lesson plans. 


Identify students who are eligible for targeted 
interventions. 


Identify strengths. 


Identify teaching and learning strategies. 


Improve classroom practice. 


12. How well does the grade-level team support your instructional goals? 


O 


o0o0°0 


Not at all 
Somewhat well 
Moderately well 
Extremely well 
Don’t know 


LITERACY TEAMS 


13. Do you currently have a literacy team in place at your school? 


O 


Yes 


O No (If no, skip to Question 16.) 
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14. Overall, rate the quality of the literacy team’s performance in the following areas. 


Performance Areas 


Poor 


Fair 


Good 


Excellent 


Not Sure 


Using assessment data to pinpoint the staff’s 
professional development needs. 


Addressing the needs of struggling readers. 


Addressing the needs of grade-level teams. 


Improving literacy instruction at your school. 


15. How well does the literacy team support your instructional goals? 


Oo Notat all 
Somewhat well 
Moderately well 
Extremely well 
Don’t know 


o0o0°0 
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QUALITY, HIGH-INTEREST MATERIALS 


Schoolwide Intervention Materials 


16. For each of the materials listed below, 


indicate how frequently you currently use the materials to teach literacy. 
For those that you are using, rate how effective they are in supporting student learning in language arts. 


Materials 


a) How frequently do you currently use the materials 


to teach literacy? 


b) For those you are currently using, rate 


howeffective the materials are in supporting student 
learning in language arts? 


N/A 


(Do Not 
Have) 


Not 
Currently 
Using 


Less 
than 
once a 
month 


1to3 
times a 
month 


1 to3 
times a 
week 


4to5 
times a 
week 


Not at 
all 


Minimal 
-ly 


Effective | Effective 


Some- 
what 


Effective 


Effective 


Very 
Effective 


Don’t 
Know 


Listening centers (in 
the classroom) 


Media centers (three 
computers and a 
printer) 


Text sets 


Software 


Handheld computers 
(palm pilots) 


Classroom library 


Vocabulary notebooks 


Textbooks 


Reading response 
notebooks 
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Library-Based Materials 


17. How often do you take your class to the library? 


O 


o0OoO0°0 


Never 

Rarely (less than once a month) 
Sometimes (at least once a month) 
Often (at least once a week) 
Almost daily or daily 


18. To what extent do the library resources support your school’s literacy curriculum? 


O 


o0o0°0 


Not at all 

To a small extent 

To a moderate extent 
To a large extent 
Don’t know 


19. How does the librarian work with you? (Check all that apply.) 


O 


Oo 
O 
O 


The librarian does not work with me. 
The librarian provides resources for class projects. 


The librarian and I collaborate on how to supplement lessons with library resources. 


Other (please specify): 
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20. How does the librarian work with your students? (Check all that apply.) 


O Does not work with my students. 
Works with students on research skills. 


Conducts read-alouds. 


o0O0°0 


events). 
Assists students with class projects. 


o0o0°0 


Other (please specify): 


Directs students to resources tied to curriculum. 


Teaches students how to navigate Internet resources. 
Guides struggling readers to summer programs. 


PROFESSIONAL DEVELOPMENT 


21. For each of the following topics, indicate: 


oO Did you receive professional developmentduring the current year? 
oO Ifso, rate the extent to which the professional development you received has improved your teaching practices 


Provides students with information about extracurricular academic activities (e.g., spelling bee, writing competitions, 


Professional Development Areas 


If YES, to what extent did the professional development improve 


i ? 
Bceely ea your teaching practice? 
No Slight Moderate Major 
No Yes 
Improvement | Improvement | Improvement | Improvement 


Building academic vocabulary 


Classroom libraries 


Creating literacy-rich classroom environments 


Differentiating instruction 


Direct vocabulary instruction 


Guided reading 


Incorporating text sets in your instruction 


Increasing student motivation 


Lesson planning 
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Professional Development Areas 


If YES, to what extent did the professional development improve 


i ? 
stag ea your teaching practice? 
No Slight Moderate Major 
No Yes 
Improvement | Improvement | Improvement | Improvement 


Organizing the classroom to support instruction 
and practice 


P 


arent meeting 


Supporting students’ self-directed learning 


HE 


eam teaching 


sing before, during, and after reading strategies 


sing formal assessments to guide instruction 


sing informal assessments to guide instruction 


sing technology to support literacy instruction 


sing handheld computers (palm pilots) 


sing literacy-based software 


1 


U 
U 
U 
U 
U 
U 
U 
r 


sing hardware such as media centers and 
stening centers 


Using partner reading 


U 


sing the whole-part-whole classroom 


instruction model 


Working with librarians 
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ABOUT YOU 


22. What is the name of your school? 


[Drop-down list] 


ESMOND SPRY 


WACKER 


MCKINLEYPARK WALSH 
MIRELES WHISTLER 


23. Which grade level is considered your primary teaching role this year (2007-08)? 
Oo 6 

Oo 7 

Oo 8 

O Other (please specify): 
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24. What is your primary professional teaching position? (Select the single best option.) 
O General education teacher (classroom teacher) 

Bilingual/ELL teacher 

Mathematics teacher 

Reading teacher 

Reading specialist 

Science teacher 

Social studies teacher 

Special education teacher 

Other (please specify): 


OoOoO0O0O0O00 0 


25. In which of the following settings do you teach literacy? 
Self-contained 

Subject-Area specialist 

Departmentalized 
Double block 

Other (Please specify): 


oOoO000 


26. How many students are in the classroom(s) in which you teach literacy this year (2007-08)? (If you teach literacy in more than one 
classroom, provide an average.) [INSERT TEXTBOX] 


27. How many years have you been teaching? [INSERT TEXTBOX] 
28. How many years have you been teaching at this school? [INSERT TEXTBOX] 


29. How many years have you been teaching reading? [INSERT TEXTBOX] 
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Teachers will be reimbursed by CPS for their time to complete this survey. In addition, CPS will be conducting a drawing to award a 
$50 gift certificate to Barnes and Noble to one teacher from each eligible school from among those who complete both this survey and 
the SEC (details of the drawing were provided to your principal). In order to participate, we need you to identify yourself so that we 
can verify that you completed the survey. If you would like to be reimbursed and participate in the drawing, please provide your name 
and email address below, and be sure that you identified your school in Q22 above. Your survey responses will still remain strictly 
confidential and will never be reported in any form that would allow anyone to connect your responses with your name. Providing 
this information is optional. 


Please print clearly: 


Your Name: Email: 


Thank you for completing this survey! 
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Chicago Public Schools (CPS) Striving Readers 


Spring 2008 Literacy Improvement Survey for Teachers — Treatment Schools 


The following is a survey designed to gather your feedback on the essential components of the Striving Readers program. It will take 
you approximately 30 minutes to complete and results will be reported in the aggregate only. We will not use your name or identify 
individual respondents. Your feedback is extremely valuable to the success of this program. If you have questions about this survey, 
please contact Rebecca Swann at rswann@metisassoc.com or 212-425-8833. 


Comprehensive Instruction 


1. How often do you use the following practices to help struggling readers develop better reading strategies and skills? 


Practices and Techniques 


Never 


Less than 
once a 
month 


1-3 times a 
month 


1-3 times a 
week 


4-5 times a 
week 


Employing direct vocabulary instruction. 


Differentiating instruction. 


sing guided reading. 


sing PRC2 for fluency. 


sing PRC2 for comprehension. 


U 
U 
U 
U 


sing PRC2 for vocabulary development. 


Scaffolding. 


Using whole-part-whole classroom instruction model 


Establishing the purpose for reading. 


Making connections to background knowledge. 


Understanding the arrangement of text. 


Making connections between texts. 


Monitoring comprehension through questioning. 


Synthesizing information within text or across texts. 
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Don’t know 


2. How often do you use the following techniques to help struggling readers develop better reading strategies and skills? 


Techniques 


Never/ Not 
Familiar 


Less than 
once a month 


1-3 times a 
month 


1-3 times a 
week 


4-5 times a 
week 


Everybody Reads To (ERT) 


Exclusion Brainstorming 


List-Group-Label 


Predict-Locate-Add-Note (PLAN) 


ReQuest 


Interactive Notation System for Effective Reading and 
Thinking (INSERT) 


Reciprocal teaching 


ABC Graffiti 


Guided Reading and Summarizing Procedure (GRASP) 


3. Overall, how effective has the literacy intervention teacher (LIT) push-in been in improving the reading skills of struggling readers 


in your classroom? 

O Notat all effective 
Minimally effective 
Somewhat effective 
Effective 

Very effective 


o0o0°0 
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Struggling Readers: Extended Day (Afterschool) Intervention 


4. How many of your current students are involved in the Striving Readers afterschool program? 
O None (If none, skip to Question 6.) 

1 to3 

4 to6 

7 to 9 

10 or more 


o0O0°0 


5. Overall, how effective has the afterschool component been in improving the literacy abilities of struggling readers? 
oO Notat all effective 

Minimally effective 

Somewhat effective 

Effective 

Very effective 

Don’t know 


oO000 


Purposeful Assessment 


6. Does your school have a lead literacy teacher or literacy coach? 
Oo Yes 
Oo No (If no, skip to Question 9.) 


7. To what extent do you work with your lead literacy teacher/literacy coach to use assessment data for instructional planning? 
o Notat all 
Oo Toasmall extent 
oO Toamoderate extent 
o Toa large extent 
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8. Indicate how you use the data from the following assessments. (Please check all that apply.) 


Progress Assess 


Assessments Not Using | Screening | Diagnostic | Benchmarking Monitoring | Outcomes 


Learning FirstClass Views 


Learning First ClassLinks 


mClass Running Records 


Illinois Standards Achievement Test 


Basic Reading Inventory (BRI) 


Informal assessments 


Other (please specify): 


Other (please specify): 


Other (please specify): 


Data-Driven Instruction 


9. Please indicate the extent to which you use student assessment data for each of the following purposes. 


To a Moderate To a Large 


Use of Data Not at All To Some extent Fvent Extent 


Placing students in intervention programs. 


Differentiating instruction. 


Identifying skills that need to be retaught. 


Monitoring student reading progress. 


Creating instructional groups (in-class). 
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Grade-Level Teams 


10. Do you currently have grade-level teams at your school? 


11. Overall, rate the grade-level team’s ability to use classroom assessment data in the following ways. 


Oo Yes 
O No (If no, skip to Question 13.) 


Use of Data Poor 


Fair 


Good 


Excellent 


Not Sure 


Address the needs of struggling readers. 


Formalize lesson plans. 


Identify students who are eligible for targeted 
interventions. 


Identify strengths. 


Identify teaching and learning strategies. 


Improve classroom practice. 


12. How well does the grade-level team support your instructional goals? 


O 


oO0O0°0 


Not at all 
Somewhat well 
Moderately well 
Extremely well 
Don’t know 


Literacy Teams 


13. Do you currently have a literacy team in place at your school? 


O 


Yes 


O No (If no, skip to Question 16.) 


A-401 


14. Overall, rate the quality of the literacy team’s performance in the following areas. 


Performance Areas 


Poor 


Fair 


Good 


Excellent 


Not Sure 


Using assessment data to pinpoint the staff's 
professional development needs. 


Addressing the needs of struggling readers. 


Addressing the needs of grade-level teams. 


Improving literacy instruction at your school. 


15. How well does the literacy team support your instructional goals? 


Oo Notat all 
Somewhat well 
Moderately well 
Extremely well 
Don’t know 


o0o0°0 
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Quality, High-Interest Materials 


Schoolwide Intervention Materials 


16. For each of the materials listed below, 


indicate how frequently you currently use the materials to teach literacy. 
For those that you are using, rate how effective they are in supporting student learning in language arts. 


Materials 


a) How frequently do you currently use the materials 


to teach literacy? 


b) For those you are currently using, rate 


howeffective the materials are in supporting student 
learning in language arts? 


Not 
Currently 
Using 


Less 
than 
once a 
month 


1to3 
times a 
month 


4to5 
times a 
week 


Not at 
all 


Minimal 
-ly 


Effective | Effective 


Some- 
what 


Effective 


Effective 


Very 
Effective 


Don’t 
Know 


Listening centers 


Media centers (three 
computers and a 
printer) 


Text sets 


Software 


Handheld computers 
(palm pilots) 


Classroom library 


Vocabulary notebooks 


Textbooks 


Reading response 
notebooks 
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Struggling Readers Intervention Materials 


17. How familiar are you with the afterschool Accelerating Maximum Potential (AMP) program for struggling readers? 
O Not familiar (Skip to Question 19.) 
Oo Somewhat familiar 
O Moderately familiar 
o Extremely familiar 


18. To what extent are you using the Accelerating Maximum Potential (AMP) program for struggling readers in each of the following 


settings? 
Extent of use 
Settings Toasmall |Toamoderate| Toa large 
Not at all 
extent extent extent 


Afterschool literacy program 


Self-contained special education classroom 


Other (please specify): 


Library-Based Materials 


19. How often do you take your class to the library? 


O 


o0o0°0 


Never 

Rarely (less than once a month) 
Sometimes (at least once a month) 
Often (at least once a week) 
Almost daily or daily 
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20. To what extent do the library resources support the Striving Readers program? 
oO Notat all 

To a small extent 

To a moderate extent 

To a large extent 

Don’t know 


21. How does the librarian work with you? (Check all that apply.) 
Oo. The librarian does not work with me. 
The librarian provides resources for class projects. 
The librarian and I collaborate on how to supplement lessons with library resources. 
Other (please specify): 


o0°0 


22. How does the librarian work with your students? (Check all that apply.) 
O Does not work with my students. 
Works with students on research skills. 
Directs students to resources tied to curriculum. 
Conducts read-alouds. 
Provides students with information about extracurricular academic activities (e.g., spelling bee, writing competitions, 
events). 
Assists students with class projects. 
Teaches students how to navigate Internet resources. 
Guides struggling readers to summer programs. 
Other (please specify): 


o0o0°0 


o0o0°0 
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Professional Development 


23. For each of the following professional development sessions, please indicate: 
= Whether you participated, and 
"  Ifso, how useful the session was in helping you support student learning in language arts 


Professional Development Sessions 


Did you participate? 


If YES, how useful was the session? 


No 


Yes 


Not Useful 


Somewhat 
Useful 


Moderately 
Useful 


Extremely 
Useful 


AMP Intensive Intervention Program 


Summer institute 


School-year follow-up institutes 


Saturday seminars 


School-based professional development 


Graduate courses 


24. For each of the following topics, indicate: 


= Did you receive professional developmentduring the current year? 


" If so, rate the extent to which the professional development you received has improved your teaching practices. 


Professional Development Areas 


If YES, to what extent did the professional development improve 


i ? 
srecgen Ene your teaching practice? 
No Slight Moderate Major 
No Yes 
Improvement | Improvement | Improvement | Improvement 


Building academic vocabulary 


Classroom libraries 


Creating literacy-rich classroom environments 


Differentiating instruction 


Direct vocabulary instruction 


A-406 


Professional Development Areas 


If YES, to what extent did the professional development improve 


i ? 
mereived Le your teaching practice? 
No Slight Moderate Major 
No Yes 
Improvement | Improvement | Improvement | Improvement 


Guided reading 


Incorporating text sets in your instruction 


Increasing student motivation 


Lesson planning 


Organizing the classroom to support instruction 
and practice 


Parent meeting 


Supporting students’ self-directed learning 


Team teaching 


sing before, during, and after reading strategies 


sing formal assessments to guide instruction 


sing informal assessments to guide instruction 


sing handheld computers (palm pilots) 


sing literacy-based software 


sing media centers and listening centers 


U 
U 
U 
Using technology to support literacy instruction 
U 
U 
U 
U 


sing the PRC2 model 


Using the whole-part-whole classroom 
instruction model 


Working with librarians 
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About You 


25. What is the name of your school? 
[drop down list] 


26. Which grade level is considered your primary teaching role this year (2007-08)? 
Oo 6 

Oo 7 

Oo 8 

oO Other (please specify): 
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27. What is your primary professional teaching position? (Select the single best option.) 
O General education teacher (classroom teacher) 

Bilingual/ELL teacher 

Mathematics teacher 

Reading teacher 

Reading specialist 

Science teacher 

Social studies teacher 

Special education teacher 

Other (please specify): 


OoOoO0O0O0O00 0 


28. In which of the following settings do you teach literacy? 
Self-contained 

Subject-Area specialist 

Departmentalized 
Double block 

Other (Please specify): 


oOoO00°0 


29. How many students are in the classroom(s) in which you teach literacy this year (2007-08)? (If you teach literacy in more than one 
classroom, provide an average.) [INSERT TEXTBOX] 


30. How many years have you been teaching? [INSERT TEXTBOX] 
31. How many years have you been teaching at this school? [INSERT TEXTBOX] 


32. How many years have you been teaching reading? [INSERT TEXTBOX] 
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Teachers will be reimbursed by CPS for their time to complete this survey. In order to be reimbursed we need you to identify yourself 
so that we can verify that you completed the survey. If you would like to be reimbursed, please provide your name and email address 
below, and be sure that you identified your school in Q25 above. Your survey responses will still remain strictly confidential and will 
never be reported in any form that would allow anyone to connect your responses with your name. Providing this information is 
optional. 


Please print clearly: 


Your Name: Email: 


Thank you for completing this survey! 
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Chicago Public Schools 
Pre-Observation Literacy Environment Checklist 


SCHOOL: DATE: 

TEACHER: LESSON START TIME: 
CLASS/GRADE: / LESSON END TIME: 
OBSERVER: 


1. MediaCenter: How many computers and printers: 
Oo Computers 
O Printers 


Are at least 3 computers and 1 printer in working order and easily accessible to 
students for individual and small group work? 


Oin working order 


Listening Center: 


Yes || No 


O several sets of headphones/| Yes (| No 


O audio materials for use by students 


Yes 


Yes | No 


No 


O Is it easily accessible to students?/] Yes || No 


O Is it organized and in good shape? 
O Is there a checkout system in place? 
0 Variety of texts that appeal to readers of differing abilities and interests? 


No 


Yes 


No 


Yes 


No 


o Are books grouped by genre?) Yes 
o Are materials clearly labeled? (] Yes 


o Are there both NF and Fiction books? 


No 


No 


4. Text Sets Yes 11 No 

5. Handhelds Yes LU) No 

6. Other materials 
O Newspapers! Yes [| No 
O Magazines'| Yes [| No 
O Other: 


Classroom Library (books grouped by genre, leveling, a checkout system, labels) 


Yes 


Yes 
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Chicago Public Schools 
Checklist of Observation Codes 


SCHOOL: CLASS/GRADE: i 
TEACHER: OBSERVER: 
TIME (5-MINUTE INTERVAL) START: END: 
Level Code Activity Level Code Activity 
1[2[3][4[5][¢]7]/8 4/5| 6 
1-Who 5a-Type of Material 
Classroom Teacher c Literary Text It 
Reading Specialist S Informational Text i 
Other special teacher sp Poetry Pp 
Literacy Intervention Teacher lit 5b-Specific Material 
Aide a Board/Chart be 
Librarian Ib Computer — Web based wb 
No one n Computer Software cs 
Not applicable 9 Computer to write on cw 
2-Grouping Listening Center Ic 
Whole class/Large group Ww Newspapers, magazines nm 
Small Group s Other Books ob 
Pairs Pp Palm pm 
Other ° Paper and pen/pencil pp 
Not Applicable 9 Student Writing sw 
2a-Small Groups Code Text book tb 
Individual Reading ir Text sets ts 
Partner Reading pr Worksheet Ww 
Teacher Guided Reading tg Other ° 
Written Response wr Not Applicable 9 
Book Clubs be 6-Interaction Style 
MediaCenter mc Telling t 
Other ° Modeling m 
3-Major academic focus Recitation r 
Reading r Coaching/scaffolding c 
Comprehension c Listening/watching I 
Composition/Writing Ww Reading aloud ra 
Other Language | Assessment a 
Other/ Not Applicable 0/9 Discussion d 
4-Activity Other/ Not Applicable 0/9 
Reading connected text r 7-Expected Pupil Response 
Listening to connected text | Reading r 
Comprehension skill c Reading, turn-taking r-tt 
Comprehension Strategy:: Orally responding or 
Summarizing sm Oral turn-taking or-tt 
Questioning qu Listening | 
Predicting pr Writing — long response we 
Text Structure ts Writing — short response ws 
Visualizing VZ Manipulating m 
Inferencing in Multi-modal mr 
representation 
Metacognition mcg Other/ Not applicable o/9 
Writing Ww 
Vocabulary Vv 
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Word parts wp 
(Letter ID, Phonemic 

Awareness, Phonics, 

Decoding) 

Spelling S 

Grammar g 

Word recognition strategy, wr 
Word ID, Sight words 

Other/ Not applicable 0/9 


(A) 


(B) 


# of Students on Task: 


Total # of Students: 
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Observation Code Definitions 


1. MediaCenter: How many computers and printers: Computers Printers 
Are at least 3 computers and 1 printer in working order and easily accessible to 
students for individual and small group work? [| Yes || No 

2. Listening Center: in working order, several sets of headphones, audio materials for 


use by students|] Yes (| No 
3. Classroom Library (books grouped by genre, leveling, a checkout system, labels) 

OIs it easily accessible to students?/| Yes || No 

O Is it organized and in good shape?!) Yes [| No 

O Is there a checkout system in place? [| Yes [| No 

Oo Are there a variety of texts that appeal to readers of differing abilities and 


interests?L| Yes 


o Are books grouped by genre? 
o Are materials clearly labeled? 


No 


Yes || No 


Yes || No 


oO Are there both NF and Fiction books? Yes 1 No 
4. Text Sets Yes 0 No 
5. Handhelds! Yes | No 
6. Other materials 

O Newspapers! Yes |) No 

O Magazines|| Yes [| No 

O Other?_| Yes |) No 


Level 1 —Who in the classroom is providing instruction/working with students? 
Observer will need to ask about the adults in the room 


Who Codes Definition 

Classroom teacher C Classroom teacher 

Reading Specialist S Reading teacher, Title 1 teacher, reading resource 
teacher, literacy lead 

Other special teacher | sp Speech and language teacher, ESL teacher, bilingual 
teacher, Special Education teacher 

LIT li Literacy intervention teacher 

Aide a Paraprofessional or instructional aid 

Librarian lb 

No one n No one is in the room, or no one is directly working 
with the students (e.g., the students are working in 
their seats independently and no one is circulating.) 

Not applicable 9 No instruction is occurring 
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Level 2 — What instructional groupings do you see? 
If grouping patterns for an adult change during 5 minutes, code all that apply. 


Grouping Codes _| Definition 

Whole class/Large w All of the students in the class (except for 1 or 2 

group individuals working with someone else), or a group 
of more than 10 students. If there are 10 or less in 
the room, code this as a small group. 

Small Group S Students are working in 2 or more groups of 3-10 
students each. If there are more than 10 students in a 
group, call this a large group. 

Pairs p Students are working in pairs 

Other O Some other grouping practice is in place 

Not Applicable 9 None of the above seem to apply; no instruction is 


taking place 


Level 2a—If small group was selected as an instructional grouping, what activities 
are taking place in the small groups? 


Small Groups Codes __| Definition 

Individual Reading ir Students are reading independently. 

Partner Reading pr Students are reading and discussing collaboratively 
with a partner. 

Teacher Guided tg Students are grouped with others at a similar reading 

Reading level and supported by the teacher to use effective 
reading strategies. There may be “before, during, 
and after” activities where students talk about, think 
about, and read through text. 

Written Response wr Students are writing in response to text they have 
read. 

Book Clubs be A student led discussion about one book. Teachers 
use book clubs for grouping students along common 
interest or common reading level. 

MediaCenter mc Students are using the media center. 

Other 0 Other 
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Level 3 — What major academic area is being covered? 


Who Codes _| Definition 

Reading r Reading, word recognition, vocabulary, fluency 
practice. 

Comprehension c Reading comprehension, answering oral 


comprehension questions, literature 
study(discussion groups), writing response to 
reading (where this is the major purpose for the 
writing is to demonstrate comprehension) 


Composition/Writing Ww Writing for the purpose of expressing or 
communicating ideas (but not writing in which 
major purpose is to respond to reading); learning 
how to write; writers workshop, creative writing, 
report writing. 


Other Language l Aspect of language arts other than the above; 
grammar, mechanics, oral expression, etc. 

Other 0 Focus is academic but not in literacy 

Not Applicable 9 None of the above seem to apply; no instruction 


is taking place 


Level 4 — What is the specific literacy activity or activity of the classroom teacher? 
(can choose up to four, recording number of minutes as we go.) 


Code levels 5, 6, 7 at the same time for each literacy event observed during the 5-minute 
observation. 


Code each literacy activity or event that has one of the specific foci below. For example, 
students are reading silently (r) and then switch to taking about lower-level meaning of 
text (m1); students stop to talk about the meaning of a word (v); they go back to talking 
about lower-level meaning of text. Each event or activity should be coded as having one 
particular focus. Code each literacy activity only ONCE as opposed to multiple times 


29 66 


during a 5-minute segment. Hence, for the above, “r’, “m1”, and “v” would be coded. 
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Activity 


Codes 


Definition 


Reading connected text 


Students are engaged in reading text. This includes 
silent reading, choral reading, and oral turn-taking 
reading. 


Listening to connected 
text 


—_— 


Students are engaged in listening to text. (If teacher 
is reading to students, code even if students are 
following along.) 


Comprehension skill 


Comprehension activity other than a comprehension 
strategy which is at a lower level of thinking (e.g. 
traditional skill work such as identifying main idea, 
cause-effect, fact-opinion) This differs from 
comprehension strategies in that it is more likely to 
be a decontextualized lesson than tied to a particular 
text. 

Example: A lesson about identifying main idea or 
distinguishing between facts and opinions. 


Comprehension Strategy: 


Summarizing 


sm 


Students are asked to identify the topic and main 
idea of the text, and then, either verbally or in 
writing, tell (in their own words) what they just 
read. 


Comprehension Strategy: 


Questioning 


qu 


Students ask a goal setting question prior to reading, 
use question statements or words as they read (i.e., 
Who? What? Where? etc.), or ask questions that go 
beyond the text. 


Comprehension Strategy: 


Predicting 


pr 


Students preview the text and then make predictions 
about what might happen next. 


Comprehension Strategy: 


Text Structure 


ts 


Text structure is the framework that helps students 
organize their thinking about the text. Examples 
include description, sequential text, compare and 
contrast, problem and solution, or cause and effect. 


Comprehension Strategy 
: Visualizing 


VZ 


Visualizing consists of making mental pictures 
while reading in a way that helps students see “in 
their mind’s eye” what is happening in the text. 
Teachers may ask, “What pictures do these words or 
phrases create in your mind?” 


Comprehension Strategy: 


Inferencing 


Inferencing consists of using previous knowledge to 
go beyond what students already know. Students 
may be asked to use their experience or clues in the 
text to make inferences, to identify cause and effect, 
or to distinguish between fact and fiction. The 
teacher may ask, “How does what you have just 
read add to what you already know about this 
topic?” 


Comprehension Strategy: 


Metacognition 


mcg 


Students are asked to monitor their comprehension 
and use strategies that are most helpful to them. 
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Writing Ww Students are engaged in writing ideas, not words or 
a sentence or two. The focus of this task is 
comprehension 


Vocabulary Vv Students are engaged in work around the meaning of 
words. Students may record words, define or explain 
them, use symbols to represent their meaning or be 
engaged in activities with vocabulary. 


Word parts wp Students are focused on letter name and letter sound. 

(Letter ID, Phonemic They maybe learning letters names or sounds in 

Awareness, Phonics, isolation or blending sounds to make words. 

Decoding) 

Spelling S Students are focused on how to spell word(s). May 
include learning about patterns or word families. 

Grammar g Students are focused on learning to define or use the 


parts of language. They may be diagramming 
sentences, or working with a variety of sentence 


structures. 
Word recognition wr Students are asked to read words they already know 
strategy, Word ID, Sight or learned (this may include sight words). Any 
words word wall work should be coded in this category. 
Other 0 Literacy focus other than above 
Not applicable 9 None of the above apply 


Level 5 - What type of materials are the classroom teacher and students using for 
this event? 


Type of Material Codes Definition 

Literary Text It Narrative text (e.g. novel, short stories, trade books, 
realistic fiction) 

Informational Text i Informational text, trade book, reference book (e.g. 
encyclopedia, etc.), newspapers, magazines, weekly 
readers. 

Poetry p All forms of poetry (rhyming, verse, etc.) 
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Level 5b - What are the materials the classroom teacher and students are using for 


this event? 


Code for each specific literacy activity or event coded. If more than one type of material 
is used for a specific level 4 activity, code all that apply (e.g. students switch from the 


listening center to a worksheet). 


Material Codes | Definition 

Board/Chart be Board, chart, or card is being used (e.g. blackboard, 
pocket chart, hanging chart, flashcards) 

Computer — Web wb 

based 

Computer Software cs (for example, AMP) 

Computer to write on | cw Word processing 

Listening Center Ic Students are listening to books on audiotape or CD 

Newspapers, nm 

magazines 

Other Books ob 

Palm pm 

Paper and pen/pencil __| pp (for writing) 

Student Writing SW Student writing (more than words or disconnected 
sentences) is being used (finished or in progress) 

Text book tb Include science, social studies or other content areas 

Text sets ts All materials will be about one topic. Students maybe 
reading different books at different levels on one 
topic. One group of students maybe reading about one 
title, while another reads a different book on the same 
topic. 

Worksheet w Worksheet, workbook page, sheet of paper, individual 
white boards for one-word or one-sentence answers. 

Other O Something other than the above is being used 

Not Applicable 9 None of the above seem to apply 
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Level 6 — What is the interaction style being used by the classroom teacher during 
this level 4 event? 

For each literacy activity or event, code each style that is observed during the 5 minute 
period, but code each style only once. 


Interaction Style Codes | Definition 

Telling t Telling or giving students information, explaining how 
to do something 

Modeling m The teacher is coded as explicitly 


showing/demonstrating the steps of how to do 
something or how to do a process as opposed to simply 
explaining it. 

Example: The teacher is reading aloud to the students. 
Every time she places a sticky note in the book, she 
stops and makes a prediction. After reading the 
prediction, she explains to her students what 
information she used to make this prediction. 


Recitation r The teacher is coded as engaging the students in 
answering questions, or responding, usually low-level 
q-a-q-a. The purpose primarily appears to be getting 
the students to answers the questions rather than 
engaging them in a formal discussion or fostering 
independence in terms of answering questions with 
more complete thinking. 


Coaching/scaffolding | c The teacher is coded as prompting/providing support 
which will transfer to other situations as students are 
attempting to perform a strategy or activity or to 
answer a question. The teacher’s apparent purpose is 
to foster independence to get a more complete action or 
to help students elaborate on an answer (rather than to 
simply get a student to answer a question). 


Listening/watching l Teacher is listening or watching and giving feedback 
as students are engaged in activity. Do not code as 
listening if the listening is only part of recitation. 


Reading aloud ra Teacher is reading aloud to students. 


Assessment a Engaging in questioning/explaining/providing of 
directions for the purpose of assessing student 
performance. Typically this would involve record 
keeping. 


Discussion d Students engaged in a discussion, which may or may 
not be led by the teacher, in which formal conventions 
of a discussion apply. Discussion is thought-provoking, 
getting students to express their ideas. Even if led by 
the teacher, students start to offer their own ideas 
rather than simply respond to the teacher. Exchange 
may be t-s-s-s rather than t-s-t-s. 


Other O Interaction style other than what is listed above. 
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Listening or watching without giving feedback 
would be coded as “other”. 


Not Applicable 


None of the above seem to apply 
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Level 7 — Expected Students Activity 


Code all that apply. 

Type of expected pupil | Codes Definition 

response 

Reading r Students are to be reading (individually or in pairs) 

Reading, turn-taking r-tt Students in group are to be reading by taking turns 

Orally responding or Students are to be orally responding. Oral 
responding is coded when there is choral 
responding, partners sharing ideas, or a majority or 
students in the group responding at the same time 
(include choral readings) 

Oral turn-taking or-tt Students in a group either wait to be called on or 
wait to take turns as the orally respond. Recitation 
most likely would have been coded at level 6. 

Listening Students in a group are listening (and no students 
is reading or orally responding). Typically this is 
coded when the teacher is telling students 
information (at level 6) or is reading aloud to the 
students (at level 4 and 6). Audio books, on 
computer. 

Writing — long response | we Students are writing a paragraph or longer. 

Writing — short WSs Students are note taking, or writing a short answer. 

response 

Manipulating m Students are to be manipulating, using their hands 
( Examples include: any type of typing, word cards 
or letters ) 

Multi-modal mr Projects, drawing, bulletin boards, video clips, 

representation scrapbooks, computer based projects 

Other O Some form of responding other than what is listed 
above is expected. 

Not applicable 9 None of the above seem to apply 
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Chicago Public Schools 
Literacy Instruction Observation Field Notes Form 


SCHOOL: CLASS/GRADE: / 
TEACHER: OBSERVER: 


TIME INTERVAL START: 


TIME INTERVAL START: 
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Observation Summary Form 


School Teacher Date 


Overall Impressions and Emphases in this Classroom/Lesson (e.g. focus on 
literature/skills/strategies, child/curriculum, centered, integrated/discrete subjects, 


instructional balance, etc.) — 


Instruction: a) type(s) of lesson (s) and content (introduction, new content, review, 
application, b) discussion of purpose, c) clarity of explanations/directions, d) type of 
interaction with the students (e.g., telling them information, using recitation, engaging 
students in an authentic discussion, providing coaching/scaffolding, providing modeling, 
having students initiate the talk, e) encouragement of self-regulation, f) teacher 


expectations, g) instructional density — 


Grouping Practices, Auxiliary Personnel, and instructional Activities of Other Students in 


the Room — 
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Materials Used During the Observation (including the teacher being observed and the 


other students in the room) - 


Student Participation and Engagement: What were the students doing? Were they 
actively engaged in the activities? How successful were the students in achieving the 


goals of the activities? - 


Classroom Management: How did the teacher maintain academic focus during her lesson, 
maintain pupil engagement, handle behavioral disruptions, establish classroom routines 


and use organizational techniques? — 


Classroom Environment and Other: Comment on any other aspects of the environment or 


other factors you think might be helpful in analysis - 
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Pre-Observation Form for Teachers 


School Teacher Date 


1. What will you be doing during the observation? 


2. What are the purposes of the lesson that will be observed? 


3. How do today’s activities fit in with your overall reading program? 


4. Is there anything special that I need to understand about today’s activities? 
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Literacy Intervention Teacher Interview Protocol — Treatment 


Schools 

Interviewee 
isco Date: 
Interviewee Title: 

Start End 
School: Timo: Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public 


Schools Striving Readers external evaluation team. For this study, we are visiting both schools 
that are using the Striving Readers curriculum and a comparison sample of schools that are not 
using Striving Readers. Please note that we are asking many of the same questions as in the fall 
because we are interested in understanding whether there have been any changes since then.We 
are interested in learning about the literacy interventions for grades 6 through 8, so please respond 
to all of the questions as they relate to those grades only. Since we are requesting a lot of 
information and we have a limited period of time, please be as succinct as you can in your 
answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. Your responses will be kept confidential. We don't 
identify individual respondents or their schools. I would like to tape this interview to be sure I 
have recorded it accurately. Is this all right? 


1. Has your role as the Literacy Intervention Teacher (LIT) in the Striving Readers 
Initiative changed since the last fall? 


a. Ifso, how? 


We would like to know more about your use ofStriving Readers classroom-based 
intervention materials. 


2. Does your school have Listening Centers(where students can access models of fluency 
and record themselves to assess their own fluency)? Yes LINo (IF NO SKIP TO 


Q3) 


a. To what extent are you using the Listening Centers? 
Not used at all Somewhat used ____ Extensively used 


b. (If not used) Why not? 
c. (fused) How are you using them? 
) For which type of activities? 


) Are you using the Listening Centers with all students or subgroups of 
students? 


d. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 
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3. Does your school haveMedia Centers(3 computers and 1 printer in the classroom)? 
Yes (No (IF NO SKIP TO Q4) 


a. To what extent are you using the Media Centers? 
____ Not used at all ____ Somewhat used ____ Extensively used 


b. (Uf not used) Why not? 


c. (fused) How are you using them? 
fe) For which type of activities? 
) Are you using the Media Centers with all students or subgroups of 
students? 


d. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 


4. Do you and/or students in your school have Handheld Computers? (|Yes No (IF 
NO SKIP TO Q5) 


a. To what extent are you using the Handheld Computers? 
____ Not used at all ____ Somewhat used ____ Extensively used 


b. To what extent are students using the Handheld Computers? 
___ Not used at all ____ Somewhat used ____ Extensively used 


c. (If not used either by LIT or by Students) Why not? 


d. (If used) How are they being used? 
f°) For which type of activities do teachers use the Handheld Computers? 
O Are the Handheld Computers being used with all students or subgroups of 
students? 


e. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 


5. Has the Achieving Maximum Potential (AMP) intervention software been installed in 
your school’s computers? (Yes No (IF NO SKIP TO Q6) 


a. To what extent are you using the software? 
___ Not used at all ____ Somewhat used ____ Extensively used 


b. (Uf not used) Why not? 
c. (If used) How is it being used? 


0 For which type of activities? 
0 Are you using it with all students or subgroups of students? 


d. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 
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6. Have you used the Striving Readers website (chicagostrivingreaders.org)? || Yes {| No 
a. Ifso, please describe how you have used it. 
b. Ifnot, why not? 


7. Does your school have Grade-Level Teams? Yes (No (IF NO, SKIP TO Q8) 


a. Are you involved in these teams? Yes (No (IF NO, SKIP TO Q8) 


b. Please describe your role in these teams, including your role in addressing the 
needs of struggling readers. 


8. Does your school have aLiteracy Leadership Team? LYes UNo (IF NO, SKIP TO 
Q9) 


a. Are you involved in this team? “Yes (CINo (IF NO, SKIP TO Q9) 


b. Please describe your role on this team in general, including your role in addressing 
the needs of struggling readers. 


9. Please describe the structure of your after school programming for struggling readers. 
a. What is your role in the after school component of Striving Readers? 
b. What successes have you had with the after school component of Striving Readers? 


c. What challenges have you encountered with the after school component of Striving 
Readers? 


10. I would like to learn more about the work that you do with students during the regular 
school day. 


a. What kinds of needs do your students have that might impact on their literacy 
development? 


b. In what ways does your work address their needs? 
c. What types of activities and resources do you use when you meet with them? 
11. Now I want to ask you a few questions about the specifics of when you meet with 
classroom teachers to plan, prioritize and coordinate instruction, responsibilities, and 
student groupings. 
a. How often do you meet with classroom teachers? When do you have these meetings? 
___ Oncea ___ Several ____ Once ___ Several ____ Daily or 


month or less times a month a week times a week almost daily 


b. Please describe the topics you discuss when meeting with classroom teachers. 
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12. 


13. 


14. 


15. 


16. 


17. 


c. Please describe the specifics of how you work with teachers to promote Striving 
Readers related practices (differentiated instruction/grouping, use of Striving Readers 
materials, etc.). 

Describe your role and responsibilities in using student data. 
a. Which assessment data are you using?For what purposes? 
b. What other types of data are you using? For what purposes? 


Overall, what are the strengths of your school’s literacy curriculum? 


Please rate the impact you think the Striving Readers program has had on the reading 
achievement of students in general and struggling readers in particular. 


a. Reading achievement — all students 


[| No impact (Some impact 1 Moderate impact (| Large impact 
Please explain your rating. 


b. Reading achievement — struggling readers 


(| No impact (Some impact [ Moderate impact [1 Large impact 
Please explain your rating. 
Overall, what are the challenges to your school’s literacy curriculum? 
What do you or your school need to better support literacy instruction? 
Is there anything else you would like to add regarding the literacy activities in your 


school? 


Thank you for your time today. 
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Principal Interview Protocol — Control Schools 


Interviewee 
Nama: Date: 
Interviewee Title: 

Start End 
School: Time: Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public 


Schools Striving Readers external evaluation team. For this study, we are visiting both schools 
that are using the Striving Readers curriculum and a comparison sample of schools that are not 
using Striving Readers. Please note that we are asking many of the same questions as in the fall 
because we are interested in understanding whether there have been any changes since then.We 
are interested in learning about the literacy interventions for grades 6 through 8, so please respond 
to all of the questions as they relate to those grades only.This interview will take about 60 
minutes. Since we are requesting a lot of information and we have a limited period of time, please 
be as succinct as you can in your answers. You will have an opportunity to elaborate further at the 
end of the interview. 


Your responses will be kept confidential. We don't identify individual respondents or their 
schools. I would like to tape this interview to be sure I have recorded it accurately. Is this all 
right? 


1. As principal of your school, hasyour role in supporting the literacy curriculum in your 
school changed since the fall? 


a. Ifso, please explain. 
2. We would like to know more about your school’s literacy curriculum. 


a. Is your school currently participating in any formal initiative or intervention to 
improve literacy? (Yes [No 


oO Ifso, please provide the name of the initiative and briefly describe it, 
including materials and resources. 


b. Does your school have specific literacy resources for struggling readers? Yes 
LINo 


oO Ifyes, please describe. 
c. Please rate the impact you think the school’s literacy curriculum and/or literacy 


initiatives have had on the reading achievement of students in general and struggling 
readers in particular. 


oO Reading achievement — all students 


No impact [Some impact [] Moderate impact [1 Large impact 
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3. 


4. 


Please explain your rating. 

O Reading achievement — struggling readers 
No impact (Some impact [) Moderate impact (| Large impact 
Please explain your rating. 


We would like to know more about your school’s efforts to integrate literacy into the 
content areas. 


a. Please describe your school’s efforts, if any, to integrate literacy into the content 
areas. 


b. Do you have school-wide text sets (i.e., sets of reading materials of different 
structures and levels, centered around specific content area themes, designed to 
improve student literacy in other subject area classes)? 


“Yes UNo (IF NO SKIP TO Q4) 


oO Are the school-wide text sets being used in the content area classrooms? 
Social Studies 5 Not Used—why? o Used—how? 5 Don’t Know 
Science 0 Not Used—why? 0 Used—how? o Don’t Know 
Mathematics 0 Not Used—why? 0 Used—how? o Don’t Know 


c. Are non-literacy staff involved in professional development related to literacy? 
(Includes: Bilingual, SPED, Math, Science, Social Studies teachers) (1 Yes LINo 


oO. If yes, who has received professional development? Please specify staff 
positions. 
Oo What topics were covered? 


Does your school haveaLiteracy Leadership Team? (|1Yes (No (IF NO, SKIP TO Q5) 


a. Which of your staff are members of the Literacy Leadership Team? 


Grade level teacher(s) Literacy Intervention Teacher (1 Librarian 
Lead Literacy Teacher Principal ELL/ESL Teacher(s) 
Special education teacher(s) 0 Other: 


b. How oftendoes the Literacy Leadership Team meet? 


Has not met [Less than once per month [Once per month [Biweekly 0 


Weekly Several times a week or more 
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c. What role does the Literacy Leadership Teamplay at your school? 
d. How does the team address the needs of struggling readers? 
e. Does the Literacy Leadership Team use assessment data? [Yes [INo 


O What types of assessment data does the team use? For what purposes? 
O Does the team use assessment data to inform PD? If yes, how? 


f. What other data sources does the teamconsider in addressing student needs? 


5. Please describe your role and responsibilities as a principal in using student assessment 
data... 


[INTERVIEWER: If the school has a Literacy Leadership Team, add the following:] 
...in addition to the Literacy Leadership Team’s use of assessment data. 


a. Do you use assessment data? (Yes LINo(IF NO, SKIP TO Q6) 
b. When do you use assessment data? For what decisions or information needs? 


6. Does your school currently offer after school programming specifically targeting 
struggling readers? (| Yes or LINo(IF NO, SKIP TO Q7) 


a. Was the after school program in place in the fall? (Yes No 


If program was in place in the fall: 


O Have there been any changes in the structure, scheduling, and enrollment 
since the fall? Yes or 1No 
0 Ifso, please explain. 


If program was NOT in place in the fall: 


O When did the after school program start (date)? 
oO What staff members are involved and what role do they play? 
oO Are the students who attend the after school program of a specific grade or 
ability level? 

0 How many students are involved? 
oO What activities and materials do you use in the after school program? 

b. What successes has the school had with the after school component? 

c. What challenges has the school encountered with the after school component? 

7. How well is technology integrated into the literacy curriculum? Would you say it is: 


Not at all integrated (JSomewhat integrated [Thoroughly integrated 


Please explain your rating. 
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10. 


11. 


12. 


13. 


What impact would you say technology has had on the reading achievement of 
struggling readers in your school? Would you say it had: 


No impact [Some impact [| Moderate impact [i Large impact 
Please explain your rating. 
Overall, what are the strengths of your school’s literacy curriculum? 


Have you participated in professional development related to literacy this school year or 
last summer? Yes (No (IF NO, SKIP TO Q11) 


a. Who sponsored each professional development activity? 
b. What topics were covered? 


c. How useful would you say the professional development has been in providing you 
with the skills and tools needed to support your school’s literacy efforts? 


(JNot at all useful (JSomewhat useful Very useful 
Please explain your rating. 


Overall, what are the challenges to your school’s literacy curriculum? 


What does your school need to better support literacy instruction? 


Is there anything else you would like to add regarding the literacy activities in your 
school? 


Thank you for your time today. 
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Principal Interview Protocol — Treatment Schools 


Interviewee 


Nama: Date: 
Interviewee Title: 

Start End 
School: Time: Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public 


Schools Striving Readers external evaluation team. For this study, we are visiting both schools 
that are using the Striving Readers curriculum and a comparison sample of schools that are not 


using Striving Readers. Please note that we are asking many of the same questions as in the fall 
because we are interested in understanding whether there have been any changes since then.We 
are interested in learning about the literacy interventions for grades 6 through 8, so please respond 


to all of the questions as they relate to those grades only. Since we are requesting a lot of 
information and we have a limited period of time, please be as succinct as you can in your 
answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. Your responses will be kept confidential. We don't 
identify individual respondents or their schools. I would like to tape this interview to be sure I 
have recorded it accurately. Is this all right? 


1. Hasyour role in the Striving Readers Initiative changed since the beginning of the 
school year?/Yes |/No (IF NO, SKIP TO Q2) 


a. Ifso, how? 
2. Does your school haveaLiteracy Leadership Team? (|Yes |/No (IF NO, SKIP TO Q3) 


a. Which of your staff are members of the Literacy Leadership Team? 


Grade level teacher(s) Literacy Intervention Teacher 1 Librarian 
Lead Literacy Teacher Principal ELL/ESL Teacher(s) 
Special education teacher(s) 0 Other: 


b. How oftendoes the Literacy Leadership Team meet? 
Has not met [| Less than once per month [| Once per month 0 
Biweekly (1 Weekly Several times a week or more 

c. What role does the Literacy Leadership Teamplay at your school? 

d. How does the team address the needs of struggling readers? 


e. Does the Literacy Leadership Team use assessment data? [Yes [INo 
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oO What types of assessment data does the team use? For what purposes? 
O Does the team use assessment data to inform professional 
development?how? 


f. What other data sources does the teamconsider in addressing student needs? 


3. Please describe your role and responsibilities as a principal in using student assessment 
data... 


[INTERVIEWER: If the school has a Literacy Leadership Team, add the following:] 
...in addition to the Literacy Leadership Team’s use of assessment data. 


a. Do you use assessment data? Yes LINo(IF NO, SKIP TO Q4) 
b. When do you use assessment data? For what decisions or information needs? 

4. We would like to know more about theafter school programming for struggling readers. 
a. Was the after school program in place in the fall? 7Yes No 


If program was in place in the fall: 


O Have there been any changes in the structure, scheduling, and enrollment 
since the fall? 1Yes INo 
0 Ifso, please explain. 


If program was NOT in place in the fall: 


O When did the after school program start (date)? 

What staff members are involved and what role do they play? 

oO Are the students who attend the after school program of a specific grade or 
ability level? 

O How many students are involved? 

O What activities and materials do you use in the after school program? 


o) 


b. What successes has the school had with the after school component of Striving 
Readers? 


c. What challenges has the school encountered with the after school component of 
Striving Readers? 


5. We would like to know more about your school’s efforts to integrate literacy into the 
content areas. 


a. Please describe your school’s efforts, if any, to integrate literacy into the content 
areas. 


b. Do you have school-wide text sets (i.e., sets of reading materials of different 
structures and levels, centered around specific content area themes, designed to 
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improve student literacy in other subject area classes)? (Yes LINo (IF NO 
SKIP TO Q6) 
oO Are the school-wide text sets being used in the content area classrooms? 
Social Studies 5 Not Used—why? o Used—how? 0 Don’t Know 
Science o Not Used—why? o Used—how? 5 Don’t Know 
Mathematics o Not Used—why? o Used—how? o Don’t Know 


c. Are non-literacy staff involved in professional development for the Striving Readers 
project? (Includes: Bilingual, SPED, Math, Science, Social Studies teachers) Yes 
LINo 
oO If yes, who has received professional development? Please specify staff 
positions. 
oO What topics were covered? 
6. How well is technology integrated into the literacy curriculum? Would you say it is: 
(JNot at all integrated [Somewhat integrated [Thoroughly integrated 


Please explain your rating. 


7. What impact would you say technology has had on the reading achievement of 
struggling readers in your school? Would you say it had: 


(| No impact (Some impact 1 Moderate impact (1 Large impact 

Please explain your rating. 

8. Have you participated in the Striving Readers professional development related to 
literacy this school year or last summer? (Some examples of Striving Readers professional 
development include the Summer Institute, the school-year institutes, the leaders seminars, 
and the NLU coursework) “Yes (CINo (IF NO, SKIP TO Q9) 

a. How useful would you say the Striving Readers professional development has been 
in providing you with the skills and tools needed to effectively implement the 
Striving Readers program? 

(JNot at all useful (JSomewhat useful (Very useful 
Please explain your rating. 


9. Overall, what are the strengths of your school’s literacy curriculum? 


10. Please rate the impact you think the Striving Readers program has had on the reading 
achievement of students in general and struggling readers in particular. 


a. Reading achievement — all students 
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(| No impact (Some impact [| Moderate impact [1 Large impact 
Please explain your rating. 


b. Reading achievement — struggling readers 


(| No impact (Some impact [ Moderate impact [1 Large impact 
Please explain your rating. 


11. Overall, what are the challenges to your school’s literacy curriculum? 
12. What does your school need to better support literacy instruction? 
13. Is there anything else you would like to add regarding the literacy activities in your 


school? 


Thank you for your time today. 
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Lead Literacy Teacher Interview Protocol — Control Schools 


Interviewee 
Nisa Date: 
Interviewee Title: 

Start End 
School: Time: Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public 


Schools Striving Readers external evaluation team. For this study, we are visiting both schools 
that are using the Striving Readers curriculum and a comparison sample of schools that are not 
using Striving Readers. Please note that we are asking many of the same questions as in the fall 
because we are interested in understanding whether there have been any changes since then.We 
are interested in learning about the literacy interventions for grades 6 through 8, so please respond 
to all of the questions as they relate to those grades only. Since we are requesting a lot of 
information and we have a limited period of time, please be as succinct as you can in your 
answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. Your responses will be kept confidential. We don't 
identify individual respondents or their schools. I would like to tape this interview to be sure I 
have recorded it accurately. Is this all right? 


1. Please describe your role as [state interviewee’s title] at your school. 


a. Has your role changed since the fall? 1Yes (No 
oO Ifso, how? 


2. We would like to know more about your school’s grade 6-8literacy curriculum. 


a. Is your school currently participating in any formal initiative or intervention to 
improve literacy? (Yes [No 
oO Ifso, please provide the name of the initiative and describe it briefly, 
including materials and resources used. 


b. Does your school have specific resources for struggling readers? “Yes No 
oO Ifyes, please describe. 


c. Please rate the impact you think the school’s literacy curriculum and/or literacy 
initiatives have had on the reading achievement of students in general and struggling 
readers in particular. 


O Reading achievement — all students 


[| No impact (Some impact 1 Moderate impact (1 Large impact 
Please explain your rating. 


oO Reading achievement — struggling readers 
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(| No impact (Some impact [ Moderate impact [1 Large impact 
Please explain your rating. 


We would like to know more about literacy-based intervention materials that are being 
used in the classrooms to support literacy instruction. 


3. Do grade 6-8 literacyteachers in your school have Listening Centers (where students can 
access models of fluency and record themselves to assess their own fluency)? Yes 
No (IF NO SKIP TO Q4) 


a. To what extent are teachers using the Listening Centers? 
___ Not used at all ___ Somewhat used ____ Extensively used 


b. (Uf not used) Why not? 


c. (If used) How are they being used? 
fe) For which type of activities do teachers use the Listening Centers? 
O Are the Listening Centers being used with all students or subgroups of 
students? 


d. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 


4. Do grade 6-8 literacyteachers in your school haveMedia Centers(computers and printer 
in the classroom)? (1Yes_ [INo (IF NO SKIP TO Q5) 


a. To what extent are teachers using the Media Centers? 
___ Not used at all ___ Somewhat used ____ Extensively used 


b. (Uf not used) Why not? 
c. (If used) How are they being used? 
0) For which type of activities do teachers use the Media Centers? 


oO Are the Media Centers being used with all students or subgroups of 
students? 


d. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 


5. Do grade 6-8 literacyteachers and/or studentsin your school have Handheld 
Computers? (Yes No (IF NOSKIP TO Q6) 


a. To what extent are teachers using the Handheld Computers? 
____ Not used at all ____ Somewhat used ____ Extensively used 


b. To what extent are students using the Handheld Computers? 
___ Not used at all ___ Somewhat used ____ Extensively used 
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c. (Ifnot used) Why not? 
d. (If used) How are they being used? 
O For which type of activities do teachers use the Handheld Computers? 


oO Are the Handheld Computers being used with all students or subgroups of 
students? 


e. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 


6. Does your school use literacy-based computer software designed for grade 6-8 students? 
(Yes (INo (IF NO SKIP TO Q7) 


a. Please provide a name and describe briefly. 


b. To what extent is the software being used? 
___ Not used at all ____ Somewhat used ____ Extensively used 


c. (Ifnot used) Why not? 
d. (If used) How is it being used? 


0 For which type of activities? 
O Is it being used with all students or subgroups of students? 


e. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 


7. Do grade 6-8 literacyteachers in your school use any other technology component to 
support literacy instruction? _]Yes LINo 


a. Ifso, please describe. 


8. Does your school currently offer after school programming specifically targeting 
struggling readers in grades 6-8? [Yes “INo(If NO, SKIP TO Q9) 


a. Are you involved in the afterschool programming? [1Yes LINo(If NO, SKIP TO Q10) 
b. Please describe your role and responsibilities. 
c. What successes has the school had with the after school component? 
d. What challenges has the school encountered with the after school component? 
9. Does your school have Grade-Level Teams? (Yes UNo (IF NO, SKIP TO Q10) 
a. Are you involved in those teams? [Yes ([INo (IF NO, SKIP TO Q10) 


b. What are the roles/positions of the staff members who comprise each grade-level 
team? 
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10. 


11. 


12. 


13. 


14. 


15. 


Grade level teachers( Literacy Intervention Teacher (Librarian 
Lead Literacy Teacher [ Principal 0 ESL/ELL Teacher 


Special education teacher (1 Other: 


c. How often do the grade-level teams meet? 


0 Has not met [| Less than once per month [| Once per month [1 
Biweekly Weekly (iSeveral times a week or more 
d. What are the responsibilities of the teams? 
e. How do the teams address the needs of struggling readers? 
f. What types of student data do the teams review? For what purposes? 
g. Do the grade-level teams review lesson plans? “Yes (UNo 
o Ifso, how frequently do they review lesson plans? What is their purpose in 
reviewing the plans? 


oO Ifnot, why not? 


Does your school have aLiteracy Leadership Team? LYes UNo (IF NO, SKIP TO 
Qll) 


a. Are you involved in those teams? [Yes ([INo (IF NO, SKIP TO Q11) 


b. Please describe your role in general, as well as your role as it relates to addressing the 
needs of struggling readers. 


Please describe how grade 6-8 literacyteachers in your school are using data. 
a. Which assessment data are they using?For what purposes? 
b. What other types of data are they using? For what purposes? 
Overall, what are the strengths of your school’s literacy curriculum? 
Overall, what are the challenges to your school’s literacy curriculum? 
What do you or your school need to better support literacy instruction? 
Is there anything else you would like to add regarding your school’s literacy 


curriculum? 


Thank you for your time today. 
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Lead Literacy Teacher Interview Protocol — Treatment 


Schools 

Interviewee 
Nama: Date: 
Interviewee Title: 

Start End 
School: Time: Time: 
Interviewer: 
Introduction: I'm and I am one of the interviewers with the Chicago Public 


Schools Striving Readers external evaluation team. For this study, we are visiting both schools 
that are using the Striving Readers curriculum and a comparison sample of schools that are not 
using Striving Readers. Please note that we are asking many of the same questions as in the fall 
because we are interested in understanding whether there have been any changes since then.We 
are interested in learning about the literacy interventions for grades 6 through 8, so please respond 
to all of the questions as they relate to those grades only. Since we are requesting a lot of 
information and we have a limited period of time, please be as succinct as you can in your 
answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 60 minutes. Your responses will be kept confidential. We don't 
identify individual respondents or their schools. I would like to tape this interview to be sure I 
have recorded it accurately. Is this all right? 


1. Please describe your role as the [state interviewee’s title] at your school. 


a. Has your role changed since the fall? 1Yes (No 
oO Ifso, how? 


We would like to know more about the use ofStriving Readers classroom-based intervention 
materials by the middle school grade literacy teachers in your school. 


2. Do grade 6-8 literacyteachers in your school have Listening Centers (where students can 
access models of fluency and record themselves to assess their own fluency 
Yes No (IF NO SKIP TO Q3) 


a. To what extent are teachers using the Listening Centers? 
Not used at all Somewhat used ____ Extensively used 


b. (Uf not used) Why not? 
c. (If used) How are they being used? 
0 For which type of activities do teachers use the Listening Centers? 


) Are the Listening Centers being used with all students or subgroups of 
students? Which subgroups? 
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d. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 
3. Do grade 6-8 literacyteachers in your school haveMedia Centers(i.e., 3 computers and 1 
printer)? (Yes [No (IF NO SKIP TO Q4) 


a. To what extent are teachers using the Media Centers? 
____ Not used at all ___ Somewhat used ____ Extensively used 


b. (Uf not used) Why not? 


c. (If used) How are they being used? 
O For which type of activities do teachers use the Media Centers? 
O Are the Media Centers being used with all students or subgroups of 
students? Which subgroups? 


d. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 


4. Do grade 6-8 literacyteachers and/or studentsin your school have Handheld 
Computers? (Yes No (IF NO SKIP TO Q5) 


a. To what extent are teachers using the Handheld Computers? 
___ Not used at all ____ Somewhat used ____ Extensively used 


b. To what extent are students using the Handheld Computers? 
____ Not used at all ____ Somewhat used ____ Extensively used 


c. (If not used by teachers or students) Why not? 


d. (If used) How are they being used? 
) For which type of activities are the Handheld Computers used? 
fe) Are the Handheld Computers being used with all students or subgroups of 
students? Which subgroups? 


e. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 


5. Has the Achieving Maximum Potential(AMP) intervention software been installed in 
your school’s computers? [Yes No (IF NO SKIP TO Q6) 


a. To what extent is the software being used by grade 6-8 literacy teachers in your 
school? 
____ Not used at all ____ Somewhat used ____ Extensively used 


b. (Uf not used) Why not? 
c. (fused) How is it being used? 
fe) For which type of activities? 


O Is it being used with all students or subgroups of students? Which 
subgroups? 
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d. In what ways, if any, has this technology improved instruction and student learning in 
language arts? 


6. Are you involved in the after school component of the Striving Readers? |/Yes [/No 
(IF NO, SKIP TO Q7) 


a. Please describe your role and responsibilities. 
b. What successes has the school had with the after school component? 
c. What challenges has the school encountered with the after school component? 


7. Does your school have Grade-Level Teams? Yes (No (IF NO, SKIP TO Q8) 


a. Are you involved in these teams? [Yes ([INo (IF NO, SKIP TOQ8) 


b. What are the roles/positions of the staff members who comprise each grade-level 
team? 


Grade level teachers (| Literacy Intervention Teacher (LIT)O Librarian 
Lead Literacy Teacher = Principal 0 ELL/ESL Teacher 


Special education teacher (1 Other: 


c. How often do the grade-level teams meet? 
0 Have not met [| Less than once per month [Once per month 0 
Biweekly 0 Weekly [iSeveral times a week or more 
d. What are the responsibilities of the teams? 
e. How do the teams address the needs of struggling readers? 
f. To what extent do the teams use Striving Readers materials? For what purposes? 
g. What types of student data do the teams review? For what purposes? 
h. Do the grade-level teams review lesson plans? “Yes (No 
0 Ifso, how frequently do they review lesson plans? What is their purpose in 
reviewing the plans? 


oO Ifnot, why not? 


8. Does your school have aLiteracy Leadership Team? L1Yes (LNo (IF NO, SKIP TO 
Q9) 


a. Are you involved in this team? “Yes (CINo (IF NO, SKIP TO Q9) 
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10. 


11. 


12. 


13. 


14. 


b. Please describe your role in this team, including your role as it relates to addressing 
the needs of struggling readers. 


We would like to know how grade 6-8 literacyteachers in your school are using data. 
a. Which assessment data are they using?For what purposes? 
b. What other types of data are they using? For what purposes? 

Overall, what are the strengths of your school’s grade 6-8literacy curriculum? 


Please rate the impact you think the Striving Readers program has had on the reading 
achievement of students in general and struggling readers in particular. 


a. Reading achievement — all students 


[| No impact (Some impact [ Moderate impact — Large impact 


Please explain your rating. 


b. Reading achievement — struggling readers 
(| No impact (Some impact (| Moderate impact [1 Large impact 
Please explain your rating. 
Overall, what are the challenges to your school’s grade 6-8 literacy curriculum? 
What does your school need to better support grade 6-8 literacy instruction? 
Is there anything else you would like to add regarding the implementation of the 


Striving Readers initiative? 


Thank you for your time today. 
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Librarian Interview Protocol — Control Schools 


(INTERVIEWER: If the librarian is also responsible for the technology in the school, please 
use the combined protocol] 


Interviewee 
Nama: Date: 
Interviewee Title: 

Start End 
School: Time: Time: 
Interviewer: 
Introduction.I'm and I am one of the interviewers with the Chicago Public 


Schools Striving Readers external evaluation team. For this study, we are visiting both schools 
that are using the Striving Readers curriculum and a comparison sample of schools that are not 
using Striving Readers. Please note that we are asking many of the same questions as in the fall 
because we are interested in understanding whether there have been any changes since then. We 
are interested in learning about the literacy interventions for grades 6 through 8, so please respond 
to all of the questions as they relate to those grades only. Since we are requesting a lot of 
information and we have a limited period of time, please be as succinct as you can in your 
answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 30 minutes. Your responses will be kept confidential. We don't 
identify individual respondents or their schools. I would like to tape this interview to be sure I 
have recorded it accurately. Is this all right? 


12. Do you currently work as a full-time or part-time librarian? [Full-time [Part-time 


13. Have you been endorsed as a librarian? (Yes (No 


14. Are there any other librarians on staff? {lYes {/No 


a. If yes, how many? 


15. Are there any library aides on staff? “Yes LINo 
a. If yes, how many? 


16. We would like to know more about you’re the library’s role in supporting class projects 
and literacy activities. 


a. Please describe the nature of your collaboration with individual classroom teachers. 


b. What are the major resources that you and the other librarians provide for class 
projects and literacy activities? Please list these resources. 


c. Do you play a role in supporting the integration of technology into instruction? Yes 


LINo 
oO Ifso, please describe. 
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17. 


18. 


19. 


20. 


21. 


22. 


23. 


24. 


25. 


Does your school have grade-level teams? LYes LINo 


a. Do you work with these teams? “Yes (No 


b. Ifso, please describe your role in these teams. 


Does your school have a literacy leadership team? L1Yes LINo 
a. Do you work with this team? LYes LINo 
b. Ifso, please describe your role in this team. 

Please describe the schedule of student access to the library. 


a. What are thescheduled times during the school day when students have access to the 
library? 


b. Are there open times during the regular school day and/or afterschool hours? If so, 
please describe. 


Do you or other librarians participate in anyafter school component targeting 
struggling readers? (|Yes LINo 


a. Ifso, please describe your role and responsibilities. 


Have you received any professional development related to literacy this school year or 
last summer? Yes (CINo (IF NOT, SKIP TO Q11) 


a. Who sponsored each professional development? 
b. What topics were covered? 
c. How useful do you feel theseprofessional developmentactivities were in providing 


you with the skills and tools needed to support your school’s literacy efforts? Would 
you say: 


(JNot at all useful (JSomewhat useful Very useful 
Please explain your rating. 


Does the library role in supporting the school’s literacy curriculum work well? Please 
explain. 


Overall, what are the challenges to your role in supporting the school’s literacy 
curriculum? 


What do you need to better support literacy instruction? 
Is there anything else you would like to add regarding literacy activities in your school? 


Thank you for your time. 
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Librarian Interview Protocol — Treatment Schools 


(INTERVIEWER: If the librarian is also responsible for the technology in the school, please 
use the combined protocol] 


Interviewee 
ee Date: 
Interviewee Title: 

Start End 
School: Time: Time: 
Interviewer: 
Introduction.I'm and I am one of the interviewers with the Chicago Public 


Schools Striving Readers external evaluation team. For this study, we are visiting both schools 
that are using the Striving Readers curriculum and a comparison sample of schools that are not 
using Striving Readers. Please note that we are asking many of the same questions as in the fall 
because we are interested in understanding whether there have been any changes since then.We 
are interested in learning about the literacy interventions for grades 6 through 8, so please respond 
to all of the questions as they relate to those grades only. Since we are requesting a lot of 
information and we have a limited period of time, please be as succinct as you can in your 
answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 30 minutes. Your responses will be kept confidential. We don't 
identify individual respondents or their schools. I would like to tape this interview to be sure I 
have recorded it accurately. Is this all right? 


1. Do you currently work as a full-time or part-time librarian? (JFull-time [/Part-time 


2. Have you been endorsed as a librarian? ([|Yes ([INo 


3. Are there any other librarians on staff? [Yes [INo 


a. If yes, how many? 
4. Are there any library aides on staff? “Yes LINo 
a. If yes, how many? 


5. We would like to know more about the library’s role in supporting class projects and 
literacy activities. 


a. Please describe the nature of your collaboration with individual classroom teachers. 


b. What are the major resources that you and the other librarians provide for class 
projects and literacy activities? Please list these resources. 


c. Has your relationship with them changed as a result of the Striving Readers program? 
If so, how? 
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d. Do you play a role in supporting the integration of technology into instruction? Yes 
LINo 
oO Ifso, please describe. 


6. Have you ordered any new materials as a part of the Striving Readers program? Yes 
LINo 


a. Ifso, please list and describe them. 


b. In what ways are the 6" i ge rade students using these additional materials? 
Describe. 


7. Do you work withgrade-level teams at your school? LYes LINo 
a. Ifso, please describe your role in these teams. 

8. Do you work with theliteracy leadership team at your school? ([jYes ([INo 
a. Ifso, please describe your role in this team. 

9. Please describe the library schedule. 


a. What are thescheduled times during the school day when students have access to the 
library? 


b. Are there open times during the regular school day and/or afterschool hours? If so, 
please describe. 


10. Are you involved in the after school component of the Striving Readers? [JYes [INo 
a. Ifso, please describe your role and responsibilities. 

11. Have you participated in Striving Readers professional development?(Examples include 
the summer institute, school year institutes, and other PD opportunities related to literacy 
and/or struggling readers) ([J/Yes ([INo (IF NO SKIP TO Q12) 

a. What topics were covered in these trainings? 

b. How useful do you feel the Striving Readers professional development is in 
providing you with the skills and tools needed to effectively implement the Striving 
Readers program? Would you say it is: 

(JNot at all useful (JSomewhat useful Very useful 


Please explain your rating. 


12. Does the library role in supporting the Striving Readers initiative work well? Please 
explain. 


13. Overall, what are the challenges to your role in supporting the school’s literacy 
curriculum? 
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14. What do you need to better support literacy instruction? 


15. Is there anything else you would like to add regarding literacy activities in your school? 


Thank you for your time. 
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Technology Coordinator Interview Protocol — Control Schools 


[INTERVIEWER: If the librarian also acts as the technology coordinator in the school, 
please use the combined protocol in place of this one.] 


Interviewee 
Name: Date: 


Interviewee Title: 


Start End 
School: Time: Time: 
Interviewer: 
Introduction.I'm and I am one of the interviewers with the Chicago Public 


Schools Striving Readers external evaluation team. For this study, we are visiting both schools 
that are using the Striving Readers curriculum and a comparison sample of schools that are not 
using Striving Readers. Please note that we are asking many of the same questions as in the fall 
because we are interested in understanding whether there have been any changes since then. We 
are interested in learning about the literacy interventions for grades 6 through 8, so please respond 
to all of the questions as they relate to those grades only. Since we are requesting a lot of 
information and we have a limited period of time, please be as succinct as you can in your 
answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 20 minutes. Your responses will be kept confidential. We don't 
identify individual respondents or their schools. I would like to tape this interview to be sure I 
have recorded it accurately. Is this all right? 


We would like to know more about the types of technology, if any, that are being used in the 
classrooms to support literacy instruction. 


1. Do grade 6-8 literacy teachers in your school have Listening Centers(where students can 
access models of fluency and record themselves to assess their own fluency)? Yes 
LINo (IF NO SKIP TO Q2) 
a. Are they currently inuse? [Yes (No 


b. Ifnot, why not? 


2. Do grade 6-8 literacy teachers in your school have Media Centers (computers 
andprinter) in their classrooms?(|Yes |/No (IF NO SKIP TO Q3) 


a. Are they currently inuse? [Yes [No 
b. Ifnot, why not? 


3. Do grade 6-8 literacy teachers and/or students in your school have Handheld 
Computers? L1Yes UNo (IF NO SKIP TO Q4) 


a. Are they currently inuse? [Yes (No 
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b. Ifnot, why not? 


4. Do grade 6-8 literacy teachers in your school have Literacy-Based Software? | Yes 
LINo (IF NO SKIP TO Q5) 


a. Is it currently in use? (|Yes LINo 
b. Ifnot, why not? 


5. Have you received any professional development related to literacy this school year or 
last summer? “Yes UCINo (IF NO SKIP TO Q6) 


a. Who sponsored each professional development? 
b. What topics were covered? 
c. How useful do you feel theseprofessional developmentactivities were in providing 


you with the skills and tools needed to support the school’s literacy efforts? Would 
you say the professional development has been: 


(JNot at all useful (JSomewhat useful Very useful 
Please explain your rating. 


6. What would you need to better support the integration of technology into literacy 
instruction in your school? 


7. Is there anything else you would like to add regarding the technology in your school? 


Thank you for your time. 
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Technology Coordinator Interview Protocol — Treatment 
Schools 


[INTERVIEWER: If the librarian also acts as the technology coordinator in the school, 
please use the combined protocol in place of this one.] 


Interviewee 
reac Date: 
Interviewee Title: 

Start End 
School: Timo: Time: 
Interviewer: 
Introduction.I'm and I am one of the interviewers with the Chicago Public 


Schools Striving Readers external evaluation team. For this study, we are visiting both schools 
that are using the Striving Readers curriculum and a comparison sample of schools that are not 
using Striving Readers. Please note that we are asking many of the same questions as in the fall 
because we are interested in understanding whether there have been any changes since then.We 
are interested in learning about the literacy interventions for grades 6 through 8, so please respond 
to all of the questions as they relate to those grades only. Since we are requesting a lot of 
information and we have a limited period of time, please be as succinct as you can in your 
answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 20 minutes. Your responses will be kept confidential. We don't 
identify individual respondents or their schools. I would like to tape this interview to be sure I 
have recorded it accurately. Is this all right? 


We would like to know more about the types of technology, if any, that are being used in the 
classrooms to support literacy instruction. 


1. Have grade 6-8 literacy teachers received the Listening Centers(where students can 
access models of fluency and record themselves to assess their own fluency)? Yes 
LINo UF NO SKIP TO Q2) 
a. Are they currently inuse? [Yes [No 


b. Ifnot, why not? 


2. Have grade 6-8 literacy teachers received Media Centers (3 computers and 1 printer) 
for their classrooms?||/Yes {/No (IF NO SKIP TO Q3) 


a. Are they currently inuse? [Yes [No 
b. Ifnot, why not? 


3. Have grade 6-8 literacy teachers and/or students received Handheld Computers? | | Yes 
LINo (IF NOT SKIP TO Q4) 
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a. Are they currently inuse? [Yes (No 
b. Ifnot, why not? 


Is the Achieving Maximum Potential(AMP) software installed in the school’s computers? 
(Yes (INo (IF NO SKIP TO Q5) 


a. Is it currently in use? (JYes LINo 
b. Ifnot, why not? 


What issues or barriers have you encountered in using the Striving Readers 
intervention materials in your school? 


a. Did you receive all of the materials when they were needed? 

b. Did you have enough information and training to use the materials effectively? 

c. Are there any other issues or barriers? 
Have you participated in Striving Readers professional development?(Examples include 
the summer institute, school year institutes, and other PD opportunities related to literacy 
and/or struggling readers) [Yes [INo (IF NO SKIP TO Q7) 

a. What topics were covered in these trainings? 

b. How useful do you feel the Striving Readers professional development is in 


providing you with the skills and tools needed to effectively implement the Striving 
Readers program? Would you say it is: 


(INot at all useful (1Somewhat useful Very useful 
Please explain your rating. 


What would you need to better support the integration of technology into literacy 
instruction in your school? 


Is there anything else you would like to add regarding implementation of Striving 
Readers materials in your school? 


Thank you for your time. 
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Librarian/Technology Coordinator Interview Protocol — 


Control Schools 

Interviewee 
Name: Date: 
Interviewee Title: 

Start End 
School: Time: Time: 
Interviewer: 
Introduction.I'm and I am one of the interviewers with the Chicago Public 


Schools Striving Readers external evaluation team. For this study, we are visiting both schools 


that are using the Striving Readers curriculum and a comparison sample of schools that are not 


using Striving Readers. Please note that we are asking many of the same questions as in the fall 
because we are interested in understanding whether there have been any changes since then.We 
are interested in learning about the literacy interventions for grades 6 through 8, so please respond 


to all of the questions as they relate to those grades only. Since we are requesting a lot of 
information and we have a limited period of time, please be as succinct as you can in your 
answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 45 minutes. Your responses will be kept confidential. We don't 
identify individual respondents or their schools. I would like to tape this interview to be sure I 
have recorded it accurately. Is this all right? 


The first questions in this interview relate to library’s role in supporting the curriculum. At 
the end of the interview, I will also ask you a few questions about the technology component 


in your school. 
1. Do you currently work as a full-time or part-time librarian? (Full-time [Part-time 


2. Have you been endorsed as a librarian? ([|Yes_ [INo 


3. Are there any other librarians on staff? ([J/Yes [INo 
a. IZfyes, how many? 
4. Are there any library aides on staff? LYes LINo 


a. If yes, how many? 


5. We would like to know more about you’re the library’s role in supporting class projects 


and literacy activities. 


a. Please describe the nature of your collaboration with individual classroom teachers. 


b. What are the major resources that you and the other librarians provide for class 
projects and literacy activities? Please list these resources. 
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c. Do you play a role in supporting the integration of technology into instruction? Yes 
LINo 
0 Ifso, please describe. 


6. Does your school have grade-level teams? LYes LINo 
a. Do you work with these teams? “Yes CINo 


b. Ifso, please describe your role in these teams. 


7. Does your school have a literacy leadership team? Yes LINo 
a. Do you work with this team? LYes LINo 


b. Ifso, please describe your role in this team. 
8. Please describe the library schedule. 


a. What are thescheduled times during the school day when students have access to the 
library? 


b. Are there open times during the regular school day and/or afterschool hours? If so, 
please describe. 


9. Do you or other librarians participate in anyafter school component targeting 
struggling readers? (}Yes lINo 


a. Ifso, please describe your role and responsibilities. 


10. Have you received any professional development related to literacy this school year or 
last summer? (Yes (CINo (IF NO, SKIP TO Q11) 


a. Who sponsored each professional development? 
b. What topics were covered? 
c. How useful do you feel theseprofessional developmentactivities were in providing 


you with the skills and tools needed to support your school’s literacy efforts? Would 
you say: 


(JNot at all useful (JSomewhat useful (Very useful 
Please explain your rating. 


11. Does the library role in supporting the school’s literacy curriculum work well? Please 
explain. 


12. Overall, what are the challenges to your role in supporting the school’s literacy 
curriculum? 
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The next few questions relate to your school’s use of technology to support literacy 
instruction. 


13. Do grades 6-8 literacy teachers in your school have Listening Centers(where students 


14. 


15. 


16. 


17. 


18. 


19, 


can access models of fluency and record themselves to assess their own fluency)? 
(Yes (CINo (IF NO SKIP TO Q14) 


a. Are they currently inuse? [Yes [No 
b. Ifnot, why not? 


Do grades 6-8 literacy teachers in your school have Media Centers (computers 
andprinter) in their classrooms?! Yes (No (IF NO SKIP TO Q15) 


a. Are they currently inuse? [Yes [No 


b. Ifnot, why not? 


Do grades 6-8 literacy teachers and/or students in your school have Handheld 
Computers? (Yes (No (IF NO SKIP TO Q16) 


a. Are they currently inuse? [Yes [No 
b. Ifnot, why not? 


Do grades 6-8 literacy teachers in your school have Literacy-Based Software? |lYes 
LINo (IF NO SKIP TO Q17) 


a. Is it currently in use? Yes LINo 
b. Ifnot, why not? 


What other types of technology, if any, are teachers in your school using to support 
literacy instruction? Please describe. 


What would you need to better support the integration of technology into literacy 
instruction in your school? 


Is there anything else you would like to add regarding literacy activities and/or 
technology in your school? 


Thank you for your time. 


A-459 


Librarian/Technology Coordinator Interview Protocol — 
Treatment Schools 


Interviewee 
Name: Date: 
Interviewee Title: 

Start End 
School: Time: Time: 
Interviewer: 
Introduction.I'm and I am one of the interviewers with the Chicago Public 


Schools Striving Readers external evaluation team. For this study, we are visiting both schools 
that are using the Striving Readers curriculum and a comparison sample of schools that are not 


using Striving Readers. Please note that we are asking many of the same questions as in the fall 
because we are interested in understanding whether there have been any changes since then.We 


are interested in learning about the literacy interventions for grades 6 through 8, so please respond 


to all of the questions as they relate to those grades only. Since we are requesting a lot of 


information and we have a limited period of time, please be as succinct as you can in your 


answers. You will have an opportunity to elaborate further at the end of the interview. 


This interview will take about 45 minutes. Your responses will be kept confidential. We don't 
identify individual respondents or their schools. I would like to tape this interview to be sure I 


have recorded it accurately. Is this all right? 


The first questions in this interview relate to your role as librarian in supporting the 
curriculum. At the end of the interview, I will also ask you a few questions about the 


technology component in your school. 


1. Do you currently work as a full-time or part-time librarian? (JFull-time [Part-time 


2. Have you been endorsed as a librarian? 
3. Are there any other librarians on staff? 
a. Ifyes,how many? 

4. Are there any library aides on staff? 


a. Ifyes, how many? 


5. We would like to know more about the library’s role in supporting class projects and 


literacy activities. 


Yes LINo 
Yes [CINo 
“Yes LINo 


a. Please describe the nature of your collaboration with individual classroom teachers. 


b. What are the major resources that you and the other librarians provide for class 
projects and literacy activities? Please list these resources. 
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c. Has your relationship with them changed as a result of the Striving Readers program? 
If so, how? 


d. Do you play a role in supporting the integration of technology into instruction? Yes 
LINo 
oO Ifso, please describe. 
6. Have you ordered any new materials as a part of the SR program? LYes LINo 


a. Ifso, please list and describe them. 


b. In what ways are the 6" — 8" grade students using these additional materials? 
Describe. 


7. Do you work withgrade-level teams at your school? LYes LINo 
a. Ifso, please describe your role in these teams. 

8. Do you work with theliteracy leadership team at your school? ([iYes (INo 
a. Ifso, please describe your role in this team. 

9. Please describe the library schedule. 


a. What are thescheduled times during the school day when students have access to the 
library? 


b. Are there open times during the regular school day and/or afterschool hours? If so, 
please describe. 


10. Are you involved in the after school component of the Striving Readers? Yes (INo 
a. Ifso, please describe your role and responsibilities. 

11. Have you participated in Striving Readers professional development?(Examples include 
the summer institute, school year institutes, and other PD opportunities related to literacy 
and/or struggling readers) [Yes [No (IF NO SKIP TO Q12) 

a. What topics were covered in these trainings? 

b. How useful do you feel the Striving Readers professional development is in 
providing you with the skills and tools needed to effectively implement the Striving 
Readers program? Would you say it is: 

(JNot at all useful (JSomewhat useful Very useful 
Please explain your rating. 


12. Does the library role in supporting the Striving Readers initiative work well? Please 
explain. 
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13. Overall, what are the challenges to your role as a librarian in supporting the school’s 
literacy curriculum? 


The next few questions relate to your school’s use of technology to support literacy 
instruction. 


14. Have grades 6-8 literacy teachers received the Listening Centers(where students can 
access models of fluency and record themselves to assess their own fluency)? 
“Yes (No (IF NO SKIP TO Q15) 
a. Are they currently inuse? [Yes [No 


b. Ifnot, why not? 


15. Have grades 6-8 literacy teachers received Media Centers (3 computers and 1 printer) 
for their classrooms?) Yes (INo (IF NO SKIP TO Q16) 


a. Are they currently inuse? [Yes (No 
b. Ifnot, why not? 


16. Have grades 6-8 literacy teachers and/or students received Handheld Computers? 
Yes No (IF NO SKIP TO Q17) 


a. Are they currently inuse? [Yes [No 
b. Ifnot, why not? 


17. Is the Achieving Maximum Potential(AMP) software installed in the school’s computers? 
Yes (CINo (IF NO SKIP TO Q18) 


a. Is it currently in use? (}Yes lINo 
b. Ifnot, why not? 


18. What issues or barriers have you encountered in using the Striving Readers 
intervention materials in your school? 


a. Did you receive all of the materials when they were needed? 
b. Did you have enough information and training to use the materials effectively? 
c. Are there any other issues or barriers? 


19. What would you need to better support the integration of technology into literacy 
instruction in your school? 


20. Is there anything else you would like to add regarding literacy activities and/or 
technology in your school? 
Thank you for your time. 
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Appendix B: Definitions of Year 2 Fidelity Scales 


Program fidelity scales are defined in the following tables. For each of the key program 
components,’ a table is presented that outlines (where applicable) the particular characteristics or 
“sub-components” that define the main component. For each component or subcomponent, in 
the second column the tables present specific items (interview or survey questions, observation 
codes, etc.), organized by data collection instrument, that address that sub-component. In the 
third column, scales are defined to provide a rating from each item of each instrument for each 
component by assigning scores to the applicable response scales of each item. Scores are then 
aggregated across items and instruments to create total summary scores by sub-component and 
ultimately by component. All sub-component and component scores are converted to a 10-point 
scale so that they are equally weighted, and a total fidelity score is computed as the mean of the 
six component scores. Missing data was imputed by using the mean value for all other non- 
missing cases. 


' Note that these scales combine program components 4 (purposeful assessment) and 5 (data driven instruction) into 
a single scale, since data sources that were used during Year 2 did not adequately distinguish these concepts. 
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FIDELITY OF CLASSROOM MODEL SCALES 


Component 1: Reading comprehension instruction for whole school, blended intervention 


Sub-Components 


Individual/Summary Items 


Scores 


Sub-Component 1: Whole-part- 
whole instructional framework 


LIST Survey 


Q1. How often do you use the following practices to help struggling 
readers develop better reading strategies and skills? 


e Using whole-part-whole classroom instruction model 


Never=0; Less than once a month=2.5; 1-3 times a 
month=5; 1-3 times a week=7.5; 4-5 times a week=10; 
Don’t know=0 


Total Score WPW 


Sum of the Above Items/1= Range from 0 to 10 


Sub-Component 2: Use of 
gradual release model to provide 
direct, explicit instruction and 
scaffold learning for students. 


LIST Survey 


Q1. How often do you use the following practices to help struggling 
readers develop better reading strategies and skills? 


° Scaffolding 


e Differentiating Instruction 
° Using Guided Reading 
e Monitoring comprehension through questioning. 


Never=0; Less than once a month=2.5; 1-3 times a 
month=5; 1-3 times a week=7.5; 4-5 times a week=10; 
Don’t know=0 


Total Gradual Release Model Setting Score 


Sum of the Above/4 (Scale from 0 to 10) 


Sub-Component 3: Instruction 
anchor for all classrooms and 
content areas is focused on 
comprehension. 


LIST Survey 


Q1. How often do you use the following practices to help struggling 
readers develop better reading strategies and skills? 

Using guided reading. 

Using PRC2 for comprehension. 

Making connections to background knowledge. 

Understanding the arrangement of text. 

Making connections between texts. 

Monitoring comprehension through questioning. 

Synthesizing information within text or across texts. 


Never=0; Less than once a month=2.5; 1-3 times a 
month=5; 1-3 times a week=7.5; 4-5 times a week=10; 
Don’t know=0 


Q2. How often do you use the following techniques to help struggling 
readers develop better reading strategies and skills? 
° Everybody Reads To (ERT) 


Never/ Not Familiar=0; Less than once a month=2.5; 


1-3 times a month=5; 1-3 times a week=7.5; 4-5 times a 
week=10 


Exclusion Brainstorming 
List-Group-Label 
Predict-Locate-Add-Note (PLAN) 
ReQuest 
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Sub-Components 


Individual/Summary Items 


Scores 


° Interactive Notation System for Effective Reading and Thinking 
(INSERT) 

e Reciprocal teaching 

° ABC Graffiti 

e Guided Reading and Summarizing Procedure (GRASP) 


Total Systematic Comprehension Score 


Sum of the Above/16 (Scale from 0 to 10) 


Sub-Component 4: PRC2 
instructional framework, text 
sets, and technology are used 
fluidly and alternately to 
support differentiated 
instruction and increase student 
motivation, engagement, and 
understanding. 


LIST Survey 


Q1. How often do you use the following practices to help struggling 
readers develop better reading strategies and skills? 

e Differentiating instruction 

e Using PRC2 for fluency. 

e Using PRC2 for comprehension. 

e Using PRC2 for vocabulary development. 


Never=0; Less than once a month=2.5; 1-3 times a 
month=5; 1-3 times a week=7.5; 4-5 times a week=10; 
Don’t know=0 


Q16a. For each of the materials listed below, indicate how frequently 
you currently use the materials to teach literacy. 

e Listening centers 

e Media centers (three computers and a printer) 

e §=6 Text sets 

e Software 


N/A (Do Not Have)=0; Not Currently Using=0; Less than 
once a month=2.5; 1 to 3 times a month=5; | to 3 times a 
week=7.5; 4 to 5 times a week=10 


Q16b. For each of the materials listed below, for those that you are 
using, rate how effective they are in supporting student learning in 
language arts. 

e Listening centers 

e Media centers (three computers and a printer) 

e §=6 Text sets 

e Software 


Not at all Effective=0; Minimally Effective=2.5; Somewhat 
Effective=5; Effective=7.5; Very Effective=10; Don’t 
Know= missing 


Principal Interview 


Q5. Do you have school-wide text sets (i.e., supplemental reading 
materials designed to improve student literacy in other subject area 
classes) 


Yes=10; No=0 


Q5b. Are the school-wide text sets being used in the content area 
classrooms? 

oO Social Studies 

Oo Science 

oO Mathematics 


Not Used=0; Used=10; Don’t Know=missing 


Pre-Observation Literacy Environment Checklist 
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Sub-Components 


Individual/Summary Items 


Scores 


QI. Media Center: How many computers? 


and printers? 


3 or more = 10, 2 = 6.66, 1 = 3.33, 0=0; 


1 or more=10, 0=0. 


Qlb. Are at least 3 computers and | printer in working order and easily 


accessible to students for individual and small group work? a caY 
Q2a. Listening Center: In working order? 

Q2b. Listening Center: Several sets of headphones? Yes=10; No=0 
Q2c. Listening Center: Audio materials for use by students? 

Q4. Text Sets Yes=10; No=0 


Total PRC2 Score 


(Sum of above items)/23 items = Range from 0 to 10 


Sub-Component 5: Systematic 
approach to teaching academic 
content vocabulary in all 
subjects using Robert 
Marzano’s Building Academic 
Content Vocabulary 


Classroom Observation Codes 


Any of the following codes: 

4-Activity 

Vocabulary (v) 

Word parts (Letter ID, Phonemic Awareness, Phonics, Decoding) (wp) 
Word recognition strategy, Word ID, Sight words (wr) 


Combined with one of the following: 
6-Interaction Style 

Telling (t) 

Modeling (m) 

Coaching/scaffolding (c) 


Proportion of class time that includes evidence of direct 
instruction in Vocabulary 


Average proportion across all ELA class observations 
within a school 


Score = (Average Proportion)/3, max = 10 
(1.e., maximum score for 30% or more class time on 
vocabulary) 


LIST Survey 


Q1 How often do you use the following practices to help struggling 
readers develop better reading strategies and skills? 
Employing direct vocabulary instruction. 


Never=0; Less than once a month=2.5; 1-3 times a 
month=5; 1-3 times a week=7.5; 4-5 times a week=10; 
Don’t know=0 


Q16 For each of the materials listed below, for those that you are 
using, rate how effective they are in supporting student learning in 
language arts. 

Vocabulary notebooks 


Not at all Effective=0; Minimally Effective=2.5; Somewhat 
Effective=5; Effective=7.5; Very Effective=10; Don’t 
Know= missing 


Total Vocabulary Score 


Sum of the Above/3 (Score from 0 to 10) 


Total Component | - Blended Intervention Score: 


(Sum of above Sub-component Scores)/5 = Range from 
0 to 10 
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Component 2: Reading comprehension instruction for targeted intervention model for Tier 2 and Tier 3 students” 


Sub-Components 


Individual/Summary Items 


Scores 


Sub-Component 6: Teachers and 
Literacy Intervention Teachers 
collaboration in instructional 


planning and progress 
monitoring. 


LIT Interview 


you meet with classroom teachers to plan, prioritize and 
instruction, responsibilities, and student groupings. 
e How often do you meet with classroom teachers? 


Q11 Now I want to ask you a few questions about the specifics of when 


coordinate 


Once a month or less=2; Several times a month=4; 
Once a week=6; Several times a week=8; Daily or 
almost daily=10 


Total Collaboration Score 


Scale from 2 to 10 


Sub-Component 7: Explicit 
instruction in small group 


setting for Tier 2-3 students for 
approximately 20-30 minutes 


per day, in 7 core 
comprehension strategies: 
summarization, predicting, 
inferring, metacognition, 


visualization, questioning, and 


text structure. 


LIST Survey 


readers develop better reading strategies and skills? 
Using guided reading. 

Using PRC2 for comprehension.. 

Making connections to background knowledge. 
Understanding the arrangement of text. 

Making connections between texts. 

Monitoring comprehension through questioning. 

e _Synthesizing information within text or across texts. 


Q1 How often do you use the following practices to help struggling 


Never=0; Less than once a month=2.5; 1-3 times a 
month=5; 1-3 times a week=7.5; 4-5 times a week=10; 
Don’t know=0 


readers develop better reading strategies and skills? 
Everybody Reads To (ERT) 

Exclusion Brainstorming 

List-Group-Label 

Predict-Locate-Add-Note (PLAN) 

ReQuest 


(INSERT) 
e Reciprocal teaching 
e = =ABC Graffiti 
e Guided Reading and Summarizing Procedure (GRASP) 


Q2 How often do you use the following techniques to help struggling 


Interactive Notation System for Effective Reading and Thinking 


Never/ Not Familiar=0; Less than once a month=2.5; 


1-3 times a month=5; 1-3 times a week=7.5; 4-5 times 
a week=10 


Total Comprehension Score 


Sum of the Above/16 items (Score 0 to 10) 


Total Component 2 - Targeted Intervention Score: 


(Sum of above Sub-component Scores)/2 = Range 
from 0 to 10 


* Note that the second sub-component of the Targeted Intervention model, “Increased direct and supported instruction—an additional 20-30 minutes per day,” 
could not be directly measured through existing data sources and is not included in the fidelity scale. 
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Component 3: Reading comprehension instruction for intensive intervention model for Tier 3 students 


Sub-Components 


Individual/Summary Items 


Scores 


Sub-Component 8: Increased 
time—an additional 240 
minutes of direct and 
supported instruction beyond 
the intervention that occurs 
during the regular school day. 


AMP Schedule and Attendance 


Records 


Total # minutes AMP Classes should meet per week (240) * Number of 
Weeks (26) = 6240 minutes 


Total number of minutes attended by each student . 


Average attendance in AMP for the year across all 
Tier 3 students, using full annual attendance (6240 
minutes) as baseline. 


Score = (average attendance)/26, max=10 


Total Increased Time Score 


Range from 0 to 10 


Sub-Component 9: Small 
groups setting: 15 to 1 teacher 
student ratio. 


AMP Enrollment Records 


Number of Students Per Class 


(# Teachers)/(# Students) X 150, max=10 


Total Small Group Setting Score 


Scale from 0 to 10 


Classroom Observations of AMP After-School Program 


Sub-Component 10: Explicit 
and systematic instruction in 
seven core comprehension 
strategies: summarization, 
predicting, inferring, 
metacognition, visualization, 
questioning, and text 
structure (strategies 
introduced one at a time) 
during the additional 240 
minutes of supported 
instruction. 


Total Systematic Comprehension Score 


Sub-Component 11: Teaching 
of high volume and depth of 
academic vocabulary. 


Total Vocabulary Score 


Sub-Component 12: Guided 
fluency practice. 


Total guided fluency practice score 


(Only 10 observations were conducted of AMP 
classes during Wave 3 because of scheduling 
difficulties. As a result, these data were considered 
a non-representative sample of AMP classes and 
were excluded from the Component 3 score. ) 


Total Component 3 - Intensive Intervention Score: 


(Sum of above Sub-component Scores)/2 = Range 
from 0 to 10 
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Component 4: Frequent, purposeful assessment and adjustment of instruction with screening, diagnostic, and progress- 
monitoring tools and data-driven instruction structured through a team-based system of leadership and support 


Components 


Individual/Summary Items 


Scores 


Purposeful assessments 
with screening, diagnostic, 
and progress-monitoring 
tools and data-driven 
instruction structured 
through a team-based 
system of leadership and 
support 


Principal Interviews 


Q2e Does Literacy Leadership Team use assessment data? 


Q3a Do you use assessment data? 


Not Used=0; Used=10 


LIST Surveys 


Q6/7 Does your school have a lead literacy teacher or literacy coach? If YES: To 
what extent do you work with your lead literacy teacher/literacy coach to use 
assessment data for instructional planning? 


No coach or Not at all=0; To a small extent=3.3; To a 
moderate extent=6.7; To a large extent=10 


Q8 Indicate how you use the data from the following assessments: 


[From Not used=0 to Used in all intended ways=10. No 
extra points for additional applications. ] 


a) ClassViews 


Not using=0; Benchmarking=+5; Assess outcomes=+5 


c) mClass Running Records 


Not Using=0; Diagnostic=+5; Progress monitoring=+5 


d) ISAT 


Not Using=0; Outcome=10 


e) BRI 


Not Using=0; Diagnostic=10 


f) Informal assessments 


Not Using=0; Progress monitoring=10 


Q9a-e Indicate extent you use student assessment data for each of the following 
purposes: 

a) Placing students in intervention programs; 

b) Differentiating instruction; 

c) Identifying skills that need to be re-taught; 

d) Monitoring student reading progress; 

e) Creating instructional groups 


Not at all=0; To a small extent=3.3; To a moderate 
extent=6.7; To a large extent=10 


Q10/Q11a-f Do you currently have grade-level teams at your school? If YES: 
Overall, rate the grade-level team’s ability to use classroom assessment data in the 
following ways: 

a) Address the needs of struggling readers; 

b) Formalize lesson plans; 

c) Identify students who are eligible for targeted interventions; 

d) Identify strengths; 

e) Identify teaching and learning strategies 

f) Improve classroom practice 


No grade-level team= 0 on all items; or Poor=0; 
Fair=3.3; Good=6.7; Excellent=10; Not sure=missing 


Q13/14 Do you currently have a literacy team in place at your school? If YES: 
Overall, rate the quality of the literacy team’s performance in: 
Using assessment data to pinpoint the staff's professional development needs. 


No literacy team= 0; or Poor=0; Fair=3.3; Good=6.7; 
Excellent=10; Not sure=missing 


Total Score: Component 4 - Purposeful Assessments 


(Sum of above items)/20 items = Range from 0 to 10 
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Component 5: High-quality, high-interest materials 


Components 


Individual/Summary Items 


Scores 


Highly motivating reading 
materials integrated with 
engaging technology and 
audio resources. 


Principal Interviews 


Q5. Do you have school-wide text sets (i.e., supplemental reading materials 
designed to improve student literacy in other subject area classes) 


Yes=10; No=0 


Q5b. Are the school-wide text sets being used in the content area 
classrooms? 


e Social Studies 


e Science 


No or Not Used=0; Used=10; Don’t 


e Mathematics 


Know=Missing 


Pre-Observation Literacy Environment Checklist 


Q1. Media Center: How many computers? 


and printers? 


3 or more 


10, 2 = 6.66, 1 = 3.33, 0=0; 


1 or more=10, 0=0. 


Qlb. Are at least 3 computers and | printer in working order and easily 
accessible to students for individual and small group work? 


Yes=10; No=0 


Q2a. Listening Center: In working order? 


Q2b. Listening Center: Several sets of headphones? 


Q2c. Listening Center: Audio materials for use by students? 


Yes=10; No=0 


Q3a. Classroom library: Is it easily accessible to students? 


Q3b. Classroom library: Is it organized and in good shape? 


Q3c. Classroom library: Is there a checkout system in place? 


Q3d. Classroom library: Are there a variety of texts that appeal to readers 
of differing abilities and interests? 


Q3e. Classroom library: Are books grouped by genre? 


Q3f. Classroom library: Are materials clearly labeled? 


Q3g. Classroom library: Are there both NF and Fiction books 


Yes=10; No=0 


Q4. Text Sets 


Yes=10; No=0 


Qé6a. Other materials: Newspapers 


Q6b. Other materials: Magazines 


Yes=10; No=0 


Observation Codes Checklist 


Q5al. Type of Material: Literary Text 


Q5a2. Type of Material: Informational Text 


Q5b1. Specific Material: Board/Chart 


Q5b2. Specific Material: Computer-Web based 


Q5b3. Specific Material: Computer Software 


Q5b4. Specific Material: Computer to write on 


Q5b5. Specific Material: Listening Center 


Check=10/Not-checked=0 
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Components 


Individual/Summary Items 


Scores 


Q5b6. Specific Material: Text book 


Q5b7. Specific Material: Text sets 


Q5b7. Specific Material: Newspapers/Magazines 


LIST Surveys 


Qlé6a. For each of the materials listed below, indicate how frequently you 
currently use the materials to teach literacy. 


e Listening centers 


e Media centers (three computers and a printer) 


e Text sets 


e Software 


e Classroom library 


e Vocabulary notebooks 


e Textbooks 


e Reading response notebooks 


N/A (Do Not Have)=0; Not Currently Using=0; 
Less than once a month=2.5; 1 to 3 times a 
month=5; | to 3 times a week=7.5; 4 to 5 times a 
week=10 


Q16b. For each of the materials listed below, for those that you are using, 


rate how effective they are in supporting student learning in language arts. 


e Listening centers 


e Media centers (three computers and a printer) 


e Text sets 


e Software 


e Classroom library 


e Vocabulary notebooks 


e Textbooks 


e Reading response notebooks 


Not at all Effective=-0; Minimally Effective=2.5; 
Somewhat Effective=5; Effective=7.5; Very 
Effective=10; Don’t Know=0 


Q20. To what extent do the library resources support the Striving Readers 
program? 


Not at all=0; To a small extent=3.33; To a 
moderate extent=6.67; To a large extent=10; Don’t 
know=0 


Total Score: Component 5 - HQ Materials 


(Sum of above items)/46 = Range from 0 to 10 
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Component 6: Integrated, progressive, high-quality professional development 


Components 


Individual/Summary Items 


Scores 


Integrated, progressive, high 
quality professional development 


Professional Development Attendance Records 


The percents below refer to the percent of meetings 
attended by LIT or principals, respectively. 


LIT Weekly Meetings with Coordinators 


[<60%]=0; [60-74% ]=1; [75-89%]=2; [90-100%]53 


Principals’ Monthly Professional Development 


[<30%]=0; [30-59% ]=1; [60-79%]=2; [80-100%]=3 


The percents below refer to the session attendance rates 
averaged across teachers 


Teachers’ Summer Institute (Yearly) 


[<25%]=0; [25-50% ]=1; [51-79%]=2; [80-100%]=3 


Teachers’ Saturday Seminar (Monthly, Years 1-2) 


[<25%]=0; [25-50%]=1; [51-79%]=2; [80-100%]=3 


Teachers’ Quarterly Follow-Up Institutes 


[<25%]=0; [25-50% ]=1; [51-79%]=2; [80-100%]=3 


Total Score Component 6 - Professional Development 


Sum of the Above Items (Scale of 0 to 15)/ * 10 


Appendix C: Year 2 Fidelity Scale Results by 
School 


Table C-1 


Results of Year 2 Implementation Fidelity Scales by School 
Major Program Components 


Mean Score 
SOTO 
Component | Component | Component 4&5 Component 
Cohort . choo! 5 4 5 Data-Driven Component 5 
umber Overall : : 6 : 
Blended Targeted Intensive Instruction Materials’ Professional 
Intervention | Intervention | Intervention & Development’ 
Assessment 
4 6.1 7.5 5.2 5.7 5.9 6.9 5.3 
5 6.6 6.9 5.5 9.9 7.0 5.3 5.3 
6 7.4 7.6 5.4 9.1 7.2 7.8 7.3 
8 6.5 7.4 6.6 6.9 6.7 6.2 5.3 
11 6.5 7.4 5. 8.0 7.4 6.6 4.0 
13 7.3 7.4 7.6 7.7 8.6 6.2 6.0 
16 6.6 7.7 3.9 7.6 7.3 6.4 6.7 
17 7.4 8.8 6.4 8.5 7.5 7.3 6.0 
Cohort 19 6.0 7.6 4.1 9.0 5.8 6.6 24 
1 20 6.8 7.6 6.3 8.1 6.6 6.7 5.3 
22 6.9 8.2 4.9 8.5 7.4 6.9 5.3 
24 6.4 7.1 6.6 8.8 5.7 5.5 4.7 
27 7.0 7.7 5.8 8.4 7.3 7.4 5.3 
29 6.9 7.2 6.2 72 6.8 5.9 8.0 
30 6.5 7.3 7.6 6.6 5.8 5.8 6.0 
31 7.1 7.9 5.9 8.0 6.7 6.8 7.3 
Cohort 1 6.7 7.6 5.9 8.0 6.9 6.5 5.7 
1 6.2 7.3 7.8 7.7 5.0 5.6 4.0 
2 6.4 6.6 5.1 7.2 6.8 5.8 6.7 
3 6.6 6.7 6.3 7.9 6.3 5.2 7.3 
7 5.9 6.9 6.1 4.9 7.6 5.4 4.7 
9 6.3 7.0 5.5 6.5 6.7 6.2 6.0 
10 6.4 7.1 5.7 6.4 6.4 6.7 6.0 
12 5.7 7.0 6.2 6.1 5.9 5.0 4.0 
Cohort 14 6.9 1D 5.6 7.7 6.6 6.9 7.3 
2 15 7.4 8.5 7.8 6.3 8.0 6.4 7.3 
18 7.0 7.6 5.9 Wed) 7.1 7.2 6.7 
21 6.2 6.7 4.2 7.8 6.0 5.6 6.7 
23 6.6 6.8 6.4 7.5 8.3 6.1 4.7 
25 6.7 7.4 4.4 8.6 6.5 6.5 6.7 
26 6.1 5.8 6.0 8.4 6.8 5.8 4.0 
28 6.7 6.7 5.2 9.1 7.3 5.9 6.0 
Cohort 2 6.5 7.1 5.9 7.3 6.8 6.0 5.8 
Overall 6.6 7.3 5.9 7.8 6.8 6.3 5.8 


* Separate tables are not provided for these components because they did not include any sub-components. 
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Table C-2 


Results of Year 2 implementation fidelity scales by school 
Component 1: Blended intervention 


Mean Score 
Comoonent Sub- Sub- Sub- Sub- Sub- 
School 2 Component | Component Component Component Component 
Cohort 1 2 3 4 5 
Number ae 
Blended Whole-Part- a ne Comprehension PRC? Marzano’s 
Intervention Whole e Seer. Focus Vocabulary 
odel 
4 75 9.6 8.0 4.4 7.7 7.6 
5 6.9 75 8.4 5.0 5.9 78 
6 7.6 8.9 8.4 49 8.5 73 
8 74 9.1 8.9 5.3 6.9 7A 
11 74 9.7 7.7 49 7.7 7.0 
13 74 10.0 9.1 522 71 5.4 
16 7.7 9.1 8.8 5.9 7A 7.6 
Ghat 17 8.8 10.0 9.5 6.2 8.6 9.4 
sr 19 7.6 93 8.0 4.2 7 8.7 
20 7.6 8.5 8.6 6.0 7A 79 
22 8.2 10.0 8.7 5.8 79 8.7 
24 7A 10.0 9.4 5.2 6.3 49 
27 7.7 8.8 8.8 5.6 74 8.1 
29 7.2 8.8 8.8 4.4 6.3 7.6 
30 73 9.2 8.1 52 6.5 74 
31 79 79 8.7 5.7 8.5 8.6 
Cohort 1 7.6 91 8.6 5.3 7.3 7.6 
1 23 9.4 8.8 5.6 5.9 6.7 
2 6.6 10.0 8.8 4.2 6.2 3.8 
3 6.7 6.3 8.2 6.0 5.6 7.6 
7 6.9 8.5 8.1 6.2 5.7 5.9 
9 7.0 8.3 8.3 5.1 6.4 7.0 
10 71 9.3 8.8 5.4 7.6 4.5 
12 7.0 8.8 8.1 4.4 6.0 79 
Cohort 14 Wiis) 9.3 8.4 5.1 7.6 7.2 
2 15 8.5 10.0 9.1 5.7 a 9.7 
18 7.6 8.0 8.8 5.7 8.2 7A 
21 6.7 8.3 8.5 4.4 6.3 6.1 
23 6.8 9.2 75 4.8 75 5.2 
25 74 8.0 79 4.8 7.6 8.5 
26 5.8 6.9 7.5 4.1 6.0 4.8 
28 6.7 9.0 8.3 4.4 6.8 4.8 
Cohort 2 7A 8.6 8.3 5.1 6.8 6.5 
Overall 7.32 7.3 8.9 8.5 5.2 71 
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Component 2: Targeted intervention 


Table C-3 
Results of Year 2 implementation fidelity scales by school 


Cohort 


Cohort 
1 


Cohort 


Mean Score 
Sub- Sub- 
School Component 2 Component Component 
Number $ Be é 
Targeted Teacher/LIT Fae Hele , 
Intervention Collaboration AE SHUC HO - 
Comprehension 

4 5.2 6.0 4.4 
5 5.5 6.0 5.0 
6 5.4 6.0 4.9 
8 6.6 8.0 5.3 
11 5.5 6.0 4.9 
13 7.6 10.0 5.2 
16 3.9 2.0 5.9 
17 6.4 6.6 6.2 
19 4.1 4.0 4.2 
20 6.3 6.6 6.0 
22 4.9 4.0 5.8 
24 6.6 8.0 5.2 
27 5.8 6.0 5.6 
29 6.2 8.0 4.4 
30 7.6 10.0 5.2 
31 5.9 6.0 Sef 
Cohort 1 5.9 6.5 5.3 
1 7.8 10.0 5.6 
2 5.1 6.0 4.2 
3 6.3 6.6 6.0 
7 6.1 6.0 6.2 
9 5.5 6.0 5.1 
10 5.7 6.0 5.4 
12 6.2 8.0 4.4 
14 5.6 6.0 5.1 
15 7.8 10.0 Sh 
18 5.9 6.0 Saf 
21 4.2 4.0 4.4 
23 6.4 8.0 4.8 
25 4.4 4.0 4.8 
26 6.0 8.0 4.1 
28 5.2 6.0 4.4 
Cohort 2 5.9 6.7 5.1 
Overall 5.87 5.9 6.6 
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Table C-4 


Results of Year 2 implementation fidelity scales by school 


Component 3: Intensive intervention 


Cohort 


Cohort 


Cohort 


Mean Score 
Sub- Sub- 
Srna Component 3 Component Component 
Number : : 
Intensive tnchedsee Small Group 
Intervention dnspucdenal Setting (15:1) 
Time 
4 5.7 1.5 10.0 
5 9.9 9.7 10.0 
6 9.1 8.9 9.4 
8 6.9 5.4 8.3 
11 8.0 6.0 10.0 
13 7.7 5.4 10.0 
16 7.6 7.8 7.3 
17 8.5 8.1 8.8 
19 9.0 8.0 10.0 
20 8.1 6.2 10.0 
22 8.5 7.0 10.0 
24 8.8 7.5 10.0 
27 8.4 6.8 10.0 
29 7.2 6.4 7.9 
30 6.6 3.2 10.0 
31 8.0 6.0 10.0 
Cohort 1 8.0 6.5 9.5 
1 Gel. 5.7 9.6 
2 12 4.4 10.0 
3 7.9 5.8 10.0 
7 4.9 1.9 7.9 
9 6.5 3.6 9.4 
10 6.4 2.8 10.0 
12 6.1 2.1 10.0 
14 Tol 5.3 10.0 
15 6.3 2.6 10.0 
18 7.5 5.0 10.0 
21 7.8 5.6 10.0 
23 75 4.9 10.0 
25 8.6 7.2 10.0 
26 8.4 6.8 10.0 
28 9.1 8.1 10.0 
Cohort 2 7.3 4.8 9.8 
Overall 7.8 7.8 6.0 
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Appendix D: Year 2 Program Implementation 
Findings from Staff Interviews 


The findings discussed herein are based on interviews with principals, LITs, LLTs, 
librarians and technology coordinators, which were conducted as part of school visits that 
took place in spring 2008. 


Component 1: Reading Comprehension Instruction for Whole-School, 
Blended Intervention 


As a result of their participation in the Striving Readers Initiative, all treatment schools 
are involved in a formal initiative to improve students’ literacy. In comparison, fewer 
control schools (54% according to principals, 45% according to LITs) are participating in 
formal literacy initiatives. Despite this variation in involvement, there were few 
differences between Striving Readers and control school staff’s responses when asked 
about their school’s efforts to integrate literacy into content areas, and concerning the 
strengths and weaknesses of their school’s literacy curricula. 


Literacy Initiatives in Control Schools 


Approximately half of the principals (54%) and LLTs (45%) at control schools reported 
that their school was participating in a formal initiative to improve students’ literacy. 
Two principals specified that their schools have Supported Core Reading Material 
Adoption (SCRMA), which provides CPS elementary schools an opportunity to adopt a 
comprehensive district-supported and endorsed PreK—5 basal reading program. Other 
programs mentioned included the Chicago Reading Initiative, READ 180, Targeted 
Intervention Plan (TIP), Glencoe Reading Series, and DePaul Connected Curriculum. 
One LLT described a grant that their school had received to support teams of teachers 
holding book discussions with students. One principal described their school’s 
partnership with National-Louis University through which classroom teachers are 
obtaining Master’s degrees in reading, and worked collaboratively with their school to 
improve literacy instruction. Several LLTs and principals described materials or 
curricula that their school was using; however, it was unclear if the materials were part of 
a formal literacy initiative. 


Integrating Literacy into Content Areas 


Principals had much to say when asked to describe their school’s efforts at integrating 
literacy into the content areas. Those from Striving Readers schools described 
professional development around integration as being important (including the summer 
workshop and other Striving Readers-related professional development), as well as 
common planning time for teachers. Others said that personnel, such as the Striving 
Readers Coach and/or LLTs, have been important in supporting teachers with integration 
and demonstrating that integration is possible. Others mentioned the PRC2 process, 
having more content area books and materials available, assigning projects that link 
literacy skills to content, AMP strategies, and school standards that cut across content 
areas. 


Similar to Striving Readers schools, several principals at control schools noted that 
professional development on the topic of literacy integration had been important and that 
their schools had purchased additional subject-related materials and books. A few 
interviewees said that vocabulary study, reading aloud, and writing had been integrated 
into science, social studies, and math classes, as had projects that tie together literacy and 
content area knowledge. In one school, the principal modeled how to teach reading in 
content area classrooms. One control school had a faculty book club that studied the use 
of literacy strategies to improve mathematics, and the club reported out to the entire 
faculty the strategies they reviewed. 


Strengths and Challenges of the Literacy Curriculum 


The responses of Striving Readers and control school staff were similar when asked 
about the strengths of the literacy curriculum. The strength of personnel was one 
common theme. Classroom teachers, literacy teachers, and coaches were praised for 
creating an environment of collaboration, for their willingness to teach and learn new 
strategies, and for their creativity and engagement. In addition, many staff noted high- 
quality materials and technology resources, the use of data to drive instruction, and 
differentiated instruction as strengths. Principals, LLTs, and LITs at Striving Readers 
schools in particular mentioned professional development as a main strength of the 
literacy curriculum. 


Challenges to the reading curriculum in both Striving Readers and control schools were 
related to the need—or perceived need—for more resources. Interviewees said that they 
would like to have more staff (e.g., more reading coaches) to enable the provision of 
individualized instruction to struggling readers. Several mentioned that they have too 
many students to facilitate a high degree of differentiation. 


Although materials and professional development were mentioned as strengths of the 
reading curriculum, some respondents from both Striving Readers and control schools 
clearly felt challenged by a need for more materials and resources, as well as for teacher 
training and support. More than one respondent characterized a lack of motivation and 
buy-in among teachers as a challenge. A shortage of time was another theme, including 
too little time for collaboration and planning among teachers and too little time in the 
classroom with students. Several interviewees mentioned parental involvement as a 
challenge and felt that it was important for parents to reinforce literacy practices at home. 


When staff from Striving Readers and control schools were asked what they or their 
schools would need to better support literacy instruction, responses reflected their 
opinions concerning the main challenges to the literacy program (described above). In 
both Striving Readers and control schools, the most common responses were: more staff, 
more training for the staff, literacy materials and technology resources, more time, and 
more parental involvement. Principals in particular tended to focus on staffing issues 
when discussing what they needed to better support literacy instruction. Control school 
principals said that they would like to have additional teachers, LLTs, literacy coaches, 
and tutors. Striving Readers principals said that they would like more librarians and 
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LITs. The responses of LLTs tended to focus on professional development as well as 
materials and resources, such as handheld computers, listening centers, books, and 
computers. They also mentioned that having additional staff would create more 
opportunities for individualized and differentiated instruction. In addition, LITs at 
Striving Readers schools mentioned that more common planning time, collaboration, and 
coordination was needed among staff. 


Component 2: Reading Comprehension Instruction for Targeted 
Intervention Model for Tier 2 and Tier 3 Students 


The LIT staff position is unique to Striving Readers schools. Because LITs are a 
resource that control schools do not have, they and the work that they perform can be 
considered a direct impact of the initiative. Interviews with LITs revealed that the 
majority of these staff members met with teachers on a weekly basis (or more often) and 
helped teachers plan lessons and literacy strategies. Other roles that they played included 
helping to facilitate assessment processes and use assessment results, working one-on- 
one or in small groups with students, and teaching or coordinating the targeted after- 
school intervention. 


Meetings with Teachers 


Almost all LITs reported meeting with classroom teachers on a regular basis. 
Specifically, 80% reported meeting with teachers at least once a week, and 14% reported 
meeting with teachers several times a month. Only 3% reported meeting with classroom 
teachers less than once a month, and another 3% of LITs said that they never met with 
teachers. 


According to many LITs, their meetings with teachers focused on reviewing and planning 
lessons, strategies, and materials. A few LITs noted that they discussed with teachers the 
needs of specific students and planned strategies for providing individualized attention to 
these students. A few also mentioned that they discussed student progress and findings 
from assessments. One LIT said that they planned extension activities to help students 
connect reading to other aspects of their life, and another said that they discussed 
behavioral issues and student grouping with teachers. 


Student Needs 


When asked to describe the needs of students that might impact literacy development, 
several LITs explained that many students are behind grade level in terms of their literacy 
development. As a consequence, these students have many needs. Those that were 
mentioned several times include the need to build vocabulary, background knowledge, 
comprehension strategies, de-coding skills, greater fluency, and pronunciation. A few 
LITs mentioned that, for many of their students, English is a second language, and these 
students need more exposure to English at home or through extracurricular activities. A 
few LITs noted that their students need support and encouragement because they have 
low motivation. 
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How the LITs Address Students’ Needs 


When describing their work and how it addresses the student needs described above, 
several LITs explained that they help to facilitate assessment processes. In particular, the 
results of assessments are used by LITs to provide appropriate support to students. A few 
LITs said that they work one-on-one or in small groups with students. Some specified 
that they help students to develop and strengthen their strategies for comprehension, de- 
coding, and vocabulary building, and others said that they model the use of 
comprehension strategies and fluency. 


Specifically related to Striving Readers practices, several LITs said that they help 
teachers to remember Striving Readers strategies and model the strategies for them. 
Strategies mentioned included whole-part-whole instruction, differentiation, PRC2, and 
grouping students. LITs also said that they discuss and review with teachers the Striving 
Readers strategies that have been covered during professional development, and help 
teachers plan how they can use these strategies in the classroom. A few LITs said that 
they help teachers think about what types of assessment strategies are appropriate to use, 
and when to administer assessments. One LIT said that every time the school receives 
Striving Readers materials, they think about how to best use the materials with students. 
One LIT said that they co-teach using Striving Readers strategies, and one allows 
teachers to observe each other using these strategies (while the LIT is covering their 
class). 


Most LITs reported that they are a teacher for the after-school component of Striving 
Readers, and many specified that they teach Grade 6 students using AMP. One LIT 
specified that they coordinate the program, including making sure that teachers have 
supplies, rotating to observe the various classes, and substituting for absent teachers as 
needed. 


Component 3: Reading Comprehension Instruction for Intensive 
Intervention Model for Tier 3 Students 


As part of the Striving Readers Initiative, all Striving Readers schools have an after- 
school component for Tier 3 students who struggle with literacy. This after-school 
component uses AMP literacy-based software. Most control schools reported that they, 
too, have after-school programming targeting struggling readers in Grades 6—8; however, 
fewer reported having literacy-based software. Staff from Striving Readers schools were 
more confident than those from control schools about the perceived successes of their 
after-school literacy program and believe the program has improved student achievement 
and literacy skills. 


Description of the After-School Component 


Compared to Striving Readers schools, which all have an after-school component, most 
principals (85%) and LLTs (85%) at control schools reported that their school offers 
after-school programming targeting struggling readers in Grades 6-8. All Striving 
Readers principals (100%) and almost all control school principals (96%) said that their 
after-school program has been in place since the fall. Similar percentages of principals 
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(38% at Striving Readers schools and 42% at control schools) reported that changes in 
the structure, scheduling, and enrollment of the after-school component have occurred 
since the fall. 


Concerning staffing of the after-school program, school librarians and LLTs from 
Striving Readers schools were less likely to be involved in the after-school program than 
librarians and LLTs in control schools. In Striving Readers schools, 12% of librarians 
and 20% of LLTs reported that they (or another librarian) participated in the after-school 
program. In control schools, these percentages were 32% and 44%, respectively. 


Successes and Challenges of the After-School Component 


When asked to describe the successes of their after-school component, several principals 
and LITs from Striving Readers schools said that student achievement and reading 
abilities have improved, as evidenced by test scores, grades, and teacher feedback. 
Several LITs specified that the program has increased students’ fluency, vocabulary, and 
comprehension. Striving Readers principals and LITs also noted that motivation and 
confidence have increased among students who participate in the program. A few 
Striving Readers school staff noted that the program created opportunities for the LIT to 
build stronger, closer relationships with students, and that some parents have also become 
engaged in the program. 


Control school principals were less certain than Striving Readers principals about the 
success of their after-school program. Several said that their program has led to 
improved achievement and test scores; however, several others said that they were not 
certain because they do not yet have data. Those who cited improvement based their 
assessment on Learning First data, grades of participating students, and the school’s 
attainment of adequate yearly progress. Similar to Striving Readers principals, a few 
control school principals said that some students and parents seem to be engaged in and 
satisfied with the program. 


Very few LLTs at Striving Readers or control schools commented on the success of the 
after-school component, although a few LLTs at control schools noted that the after- 
school program has good materials, promotes student self-esteem, and provides a safe 
environment for students. 


Concerning challenges to the after-school component, staff at both Striving Readers and 
control schools characterized attendance as the main challenge (indeed, as discussed in 
Section III of the report, increased instructional time was among the lowest rated items 
on the fidelity scales). Some specified that it is difficult to get the students who need the 
program to attend; others said that students attend the program but that attendance is 
erratic. Issues that contribute to attendance problems include a lack of parental support, 
competition with other after-school activities that students find more appealing (e.g., 
sports), transportation issues, safety concerns, and inclement weather. Discipline and 
finding enough qualified staff to teach the program were also mentioned as challenges by 
staff at both Striving Readers and control schools. 
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Several LITs at Striving Readers schools also commented that the AMP program presents 
unique challenges. They said that students find the program boring and that it doesn’t 
include incentives to motivate students. A few LITs mentioned that AMP is difficult for 
struggling readers who do not have a lot of background knowledge and are already tired 
upon arriving at the program at the end of the day. One LIT said that implementing the 
technical component of AMP is a challenge because the technology works only 
intermittently. 


AMP and Literacy-Based Software 


Most staff members at Striving Readers schools (100% of LLTs, 93% of LITs, and 86% 
of technology coordinators) reported that AMP software is installed on the school’s 
computers. In comparison, fewer staff in control schools (79% of technology 
coordinators and 47% of LLTs) reported that Grade 6—8 literacy teachers have literacy- 
based software on school computers. When asked if AMP software (or literacy-based 
software) was currently in use in their school, 80% of technology coordinators at Striving 
Readers schools and 100% of technology coordinators at control schools responded 
affirmatively. 


Reports concerning the frequency with which the software is used varied widely by staff 
position in Striving Readers schools, making it difficult to get a clear idea of the 
frequency of use. Half (50%) of LLTs reported that they or other literacy teachers use 
the AMP software extensively; the other half said that the software is used somewhat. 
However, only 11% of LITs reported that the AMP software is used extensively. 
Approximately half (52%) said that it is used somewhat, and more than one third (37%) 
said that it is not used at all. In control schools, the responses of LLTs were similar to 
those of LLTs in Striving Readers schools. Slightly more than half (56%) reported that 
literacy software is used extensively, and 44% reported that it is used somewhat. 


To gather more information about the challenges of using AMP software in Striving 
Readers schools, LLTs and LITs who reported that the software is not used extensively 
were asked why. One LLT reported a high rate of turnover among LITs in the school, 
which has resulted in difficulties with getting the new LIT trained on AMP. A few LITs 
reported having problems with computers, not having enough computers for the students 
in the program, or having difficulties with the AMP software itself. In particular, one 
LIT said that the software is not working, and one reported that the school could not 
figure out how to use the software. A few LITs reported that students do not like the 
software, including one who believed that it was too difficult for the students and one 
who said that students do not stay on task when using AMP. Two LITs mentioned that 
they use other media (including listening centers and MP3 players) instead of AMP. 


Uses of AMP and Literacy-Based Software 


LLTs and LITs were asked how AMP (or literacy-based software) was being used, 
including for which activities and by which groups or subgroups of students. In Striving 
Readers schools, LLTs and LITs reported that AMP is being used for individualized 
instruction, and to support standards, review books with students, read story selections, 
and reinforce reading skills. One LIT said that AMP software is also used for assessment 
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purposes. Most LLTs and LITs at Striving Readers schools said that AMP is being used 
with those students who attend the after-school program. 


In control schools, LLTs said that literacy-based software is used for drills and practice; 
learning games; to improve student understanding of texts; to promote comprehension, 
fluency, word knowledge, and writing; and for guided independent reading. Some LLTs 
reported that a// students use literacy-based software, and some said that only struggling 
readers use the software. 


When LLTs and LITs were asked if and how AMP or literacy-based software had 
improved instruction, responses were similar across Striving Readers and control schools. 
A few staff interviewed at Striving Readers schools said that AMP allows students to 
have more individualized instruction, and that it allows teachers to work one-on-one with 
students while others are engaged in their own work. Two staff noted that students prefer 
to work on computers than with pencil and paper, and that using computers helps to 
engage and motivate the students. A few staff said that AMP reinforces skills learned in 
the classroom. Similarly, LLTs from control schools said that literacy-based software 
allows for individualized instruction and allows teachers to work with small groups and 
one-on-one with students. One LLT from a control school said that literacy-based 
software does not improve instruction. 


LLTs in control schools only were asked to describe any other technology used to 
support literacy instruction in their school. Several mentioned overheads, LCD 
projectors, DVDs, computers, and Power Point presentations. A few mentioned 
classroom Jeopardy, smart boards, audio tapes of stories, and listening centers. 


Component 4: Frequent, Purposeful Assessment and Adjustment of 
Instruction with Screening, Diagnostic, and Progress-Monitoring Tools 


The vast majority of Striving Readers and control school principals reported that they and 
their school’s Literacy Leadership Team use assessment data to plan and differentiate 
instruction, determine when additional instruction is needed, and plan professional 
development and teacher supports. The Lead Literacy Teachers reported that literacy 
teachers use assessment data for similar purposes. 


Sources and Uses of Data 


All principals (100%) at Striving Readers schools and control schools reported using 
assessment data. In addition, almost all Striving Readers principals (97%) and all control 
school principals (100%) said that the school’s Literacy Leadership Team uses 
assessment data. When asked what types of assessment data the Literacy Leadership 
Team uses, principals at Striving Readers and control schools responded similarly— 
mentioning data from Learning First, the ISAT, the BRI, reading fluency snapshots, 
DIBELS, classroom assessments, and observations. 


Striving Readers and control school principals specified that assessment data are used to 
plan and differentiate instruction and determine areas where additional instruction is 


needed. Some principals in Striving Readers schools said that assessment data are used 
to identify students for the after-school program and to group students according to their 
literacy abilities. Most principals at Striving Readers schools responded affirmatively 
when asked if the Literacy Leadership Team uses assessment data to inform professional 
development, and they explained that assessment data helps the team pinpoint areas 
where training and support (or additional training and support) is needed. A few 
principals said that professional development has occurred around how teachers can use 
assessment data to inform their practice, and that interpreting data is often a topic 
addressed at Saturday sessions, literacy coaches’ meetings, and principals’ meetings. 


Principals were also asked what other data sources the Literacy Leadership Team 
considers in addressing student needs. Here again, responses were similar between 
Striving Readers and control schools. In both cases, many principals mentioned data 
from teacher-made exams and observations, students’ grades, assessments of student 
projects and performances, and data relating to attendance, behavior, and retention. A 
few principals mentioned that they look at students’ needs and backgrounds, including 
social and physical issues, although they did not specify their data source. One principal 
reported looking at social and emotional development data from parent reports and from 
the school nurse. Two principals in Striving Readers schools mentioned survey data, 
including parent and student surveys as well as a University of Chicago survey of parents 
and teachers that is new to CPS. 


Principals discussed when (how often) they use assessment data and for what decisions or 
informational needs, and here again, responses were similar among those from Striving 
Readers and control schools. Answers concerning frequency of data use varied 
considerably among both groups. Very few principals reported infrequent use of data, 
and many reported using it on an ongoing or even a constant basis. Some specified that 
they tend to use data a lot over the summer, at the beginning of the school year, or when 
data become available. 


Many principals in both Striving Readers and control schools described using data to 
determine students’ strengths and needs and to make decisions about instruction based on 
that information. Many said that they use data to help them make decisions about 
professional development and the types of supports that teachers need. Several 
mentioned that data are used to group students and strategize about what different groups 
need, and others said that data are used to determine teachers’ strengths and needs, to 
plan what types of books and resources are needed in the school, and to identify students 
for tutoring or special programming (for example, the after-school program). Some 
principals said that they use data to help them make decisions about classroom and 
teacher assignments, and a few use data to help determine priorities and future 
programming needs for their school, as well as decisions about promotions. 


Although answers among Striving Readers and control school principals were generally 
very similar, principals at Striving Readers schools mentioned a few uses that were not 

mentioned by those at control schools. Specifically, they said that data are used to help 
them identify Tier 3 students, for SIPAA school improvement planning, to set 
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improvement or growth goals, and to inform parents about how they can support their 
child’s learning at home. 


Data Use among Literacy Teachers 


LLTs at Striving Readers and control schools were also asked what types of assessment 
data are being used by Grade 6-8 literacy teachers, and for what purposes. LLTs at 
Striving Readers schools mentioned the ISAT, Learning First, BRI, fluency snapshots, 
and formative classroom assessments as their main data sources. These data are used in 
Striving Readers schools to pinpoint areas where students need help and to target 
instruction to those needs. Data are also used to group students, provide feedback to 
students, and measure progress. Responses among LLTs in control schools were similar; 
however, they also mentioned data from sources including reading benchmarks, extended 
response prompts, Go for the Green, and the new ELL Enlgish assessment, the Assessing 
Comprehension and Communication in English State-to-State (ACCESS) test. 


In addition to assessment data, LLTs at Striving Readers schools said that literacy 
teachers use data collected from handheld computers, data from other teachers, 
conversations with students, and their own knowledge of students. LLTs at control 
schools use data from student portfolios, the KTEA (for special education students), 
Buckle-down assessment books, teacher-made tests, and student journals, and LLTs at 
Striving Readers and control schools mentioned using data from student observations. 
LLTs at both Striving Readers and control schools use these additional data sources to 
inform and improve their instruction. 


Component 5: Data-Driven Instruction Structured through a Team- 
Based System of Leadership and Support 


All Striving Readers schools have a Literacy Leadership Team, which is a resource that 
only some control schools have. The role of these teams includes providing support to 
school staff around literacy strategies; observing classes; and planning and providing 
professional development. Staff reported that Literacy Leadership Teams at Striving 
Readers schools met more frequently than those at control schools. In addition to the 
Literacy Leadership Team, grade-level teams are another type of school resource. Most 
Striving Readers and control schools report having grade-level teams, and staff from 
Striving Readers schools reported that their teams meet less frequently than teams at 
control schools. Grade-level teams share and collaborate about strategies and materials, 
exchange information, create common lessons, map curricula, and review individual 
student work. 


Literacy Leadership Teams 


All principals, Lead Literacy Teachers, and LITs in Striving Readers schools reported 
having a Literacy Leadership Team in their school. In comparison, three quarters (76%) 
of control school principals and slightly more than two thirds of librarians (68%) and 
Lead Literacy Teachers (65%) at control schools reported having a Literacy Leadership 
Team. Almost all LITs (97%) and most Lead Literacy Teachers (90%) and librarians 
(84%) reported being involved in or working with the Literacy Leadership Team at 
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Striving Readers schools. At control schools, all Lead Literacy Teachers (100%) and the 
majority of librarians (79%) reported being involved or working with the Literacy 
Leadership Team. 


When Striving Readers principals were asked which staff members participated in the 
Literacy Leadership Team at their school, the most common answers were: the LIT (93% 
of principals gave this response), grade-level teachers (86%), the principal (69% of 
principals said that they were involved in the team), librarians, (66%), and special 
education teachers (59%). In control schools, the most common answers were: the 
principal (66% said that they were involved in the team), other* (66%), grade-level 
teachers (59%), and special education teachers (52%). Slightly less than half of 
principals in both Striving Readers and control schools (48% and 45%, respectively) said 
that the Lead Literacy Teachers was involved in the Literacy Leadership Team. 


According to principals, the Literacy Leadership Teams at Striving Readers schools met 
more frequently than those at control schools: 41% of Striving Readers principals said 
that their school’s team met weekly, another 26% said that their team met biweekly, and 
another 26% said that the team met once a month. In comparison, more than half (55%) 
of control school principals reported that their team met once a month, another 20% 
reported that it met weekly, and another 15% reported that the team met biweekly. 


Principals were queried about the role that the Literacy Leadership Team plays at their 
school. Responses among principals at Striving Readers schools were similar to but 
slightly more expansive than those at control schools. Striving Readers principals said 
that the Literacy Leadership Team in their school makes sure that the Striving Readers 
Initiative is fully implemented and well-coordinated, holds monthly book clubs (for 
teachers), discusses research-based practices related to literacy instruction, makes sure 
that teachers are well-informed about upcoming student assessments, takes part in school 
improvement planning, and evaluates the ongoing success of literacy instruction. 
Striving Readers and control school principals said that the Literacy Leadership Team 
recommends and provides support to school staff around literacy strategies. Team 
members observe classes, support teachers, and plan and provide professional 
development. Some control school principals said that the team makes decisions about 
materials needed by the school, plans literacy-focused activities, and communicates with 
parents. Some responded that the team looks at student assessment instruments and 
assessment data. 


Principals at Striving Readers and control schools responded similarly when asked how 
the Literacy Leadership Team addresses the needs of struggling readers. Many said that 
the team uses data to assess the needs of struggling readers and to offer them appropriate 
support, for example through the after-school program, tutoring, before-school assistance, 
or small-group instruction. Many principals also said that the team works with teachers 
around using strategies that are appropriate for struggling readers. Principals in Striving 
Readers schools specifically mentioned whole-part-whole instruction and differentiated 
instruction. 


* Among the “other” responses, assistant principal was most common. 
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Lead Literacy Teachers in control schools, and Lead Literacy Teachers and LITs at 
Striving Readers schools, were asked to describe their role on the Literacy Leadership 
Team. Many respondents from both Striving Readers and control schools said that they 
chaired or coordinated the team and that their role included holding team meetings, 
facilitating meetings, planning meeting agendas, and making sure that teachers attend and 
participate. Many described their role as that of an information conduit; for example, 
they share information from other meetings or from professional development sessions 
with team members and bring up for discussion issues that teachers are encountering in 
classrooms. Other roles mentioned by both Striving Readers and control school staff 
included looking at student data, making decisions about strategies and materials being 
used, reviewing instructional materials, and making sure that teachers are implementing 
research-based strategies. 


Grade-Level Teams 


All LLTs (100%) and almost all LITs (97%) at Striving Readers schools reported that 
their school has grade-level teams, and most LLTs (95%) and librarians (91%) at control 
schools reported the same. Concerning their involvement with these teams, all LLTs 
(100%), most LITs (97%), and two thirds of librarians (64%) at Striving Readers schools 
reported being involved with or working with grade-level teams. At control schools, 
89% of LLTs and 60% of librarians reported involvement with grade-level teams. 


LLTs were asked which staff members comprise grade-level teams. The most popular 
responses, which were given by at least half of LLTs at Striving Readers schools, were: 
grade-level teachers (90% of LLTs gave this response), LITs (80%), LLTs (70%), other 
(70%), the principal (60%), and special education teachers (60%). Responses among 
LLTs at control schools were slightly different. The responses given by at least half of 
LLTs at control schools were: grade-level teachers (88%), special education teachers 
(82%), the principal (65%), and other (65%). In both Striving Readers and control 
schools, literacy coaches and assistant or vice principals were common responses in the 
“other” category. 


The frequency of grade-level team meetings at Striving Readers and control schools 
varied. Half of LLTs at Striving Readers schools said that grade-level teams met weekly, 
20% reported that the teams met biweekly, 20% said they met once per month, and 10% 
said that teams met several times a week or more. In comparison, most (88%) LLTs at 
control schools said that grade-level teams met weekly, and 6% each said that teams met 
biweekly or less than once per month. 


Although half (50%) of LLTs at Striving Readers schools said that the grade-level teams 
review lesson plans, 87% of LLTs at control schools reported that the teams perform this 
function. Overall, LLTs at Striving Readers and control schools described the 
responsibilities of grade-level teams similarly. LLTs from Striving Readers schools said 
that the teams’ responsibilities include sharing and collaborating about strategies and 
materials used in the classroom; exchanging information from workshops, book talks, 
and other professional development events; creating common lessons to ensure 
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consistency across the grade level; mapping the curriculum for next year; and reviewing 
individual student work. In addition to these responsibilities, LLTs from control schools 
said that grade-level teams also plan grade-wide activities, provide updates to teachers, 
look at data, read articles, and have a study group. Only a few LLTs from Striving 
Readers schools said that grade-level teams in their school review lesson plans. They 
said this is a role that generally falls outside of the teams, usually to an assistant principal 
or other administrator. In control schools, many LLTs said that their school’s grade-level 
teams review lesson plans; however, their comments suggest that the teams talk about 
lessons or plan lessons during meetings but do not necessarily look at written lesson 
plans. 


When asked how grade-level teams address the needs of struggling readers, both groups 
of LLTs said that the teams review and assess the needs of particular students and 
determine how to address those needs, share successes and challenges in working with 
struggling readers, discuss the implementation of strategies for struggling readers, look at 
data, and share ideas about structuring student groups. Control school LLTs mentioned 
pairing struggling teachers with experienced teachers on the teams to ensure that 
modeling takes place, and said that coaching from the Literacy Coach is an important 
aspect of the teams’ efforts to address struggling readers. LLTs from Striving Readers 
schools mentioned that teams discuss differentiated instruction and how it can be used 
with struggling readers. 


Striving Readers and control school LLTs reported that grade-level teams review several 
types of data, including data from the ISAT, fluency snapshots, Learning First, teacher- 
made assessments, and extended response writing prompts. LLTs at Striving Readers 
schools also reported using data from the BRIs and observations of students. Data are 
used to identify student strengths and areas of weakness, target instruction to areas where 
students are struggling, group students, set the pace of instruction, assess progress, and 
identify students for the after-school program. 


Component 6: High-Quality, High-Interest Materials 


Striving Readers encourages the use of high-quality, high-interest materials such as text 
sets, listening centers, media centers, and handheld computers. Most Striving Readers 
and control schools reported having text sets. Interestingly, control school staff were 
slightly more likely than Striving Readers staff to report that text sets were being used in 
content area classrooms. Regarding listening centers and media centers, Striving Readers 
schools were slightly more likely than control schools to report having these resources. 
Striving Readers schools appear to be using listening centers for a greater variety of 
purposes, including differentiating instruction. Finally, although the majority of Striving 
Readers schools reported having handheld computers, very few reported that teachers and 
students are using these resources. Overall, principals in both Striving Readers and 
control schools felt that technology was somewhat integrated into their literacy 
curriculum. 


Text Sets 


Most principals at both Striving Readers (89%) and control schools (89%) reported 
having school-wide text sets. A higher percentages of control school principals reported 
that the text sets were being used in content area classrooms. For example, 88% of 
control school principals, compared to 79% of Striving Readers principals, said that text 
sets were being used in social studies classrooms, 88% of control school principals and 
71% of Striving Readers principals said that text sets were being used in science 
classrooms, and 65% of control school principals and 57% of Striving Readers principals 
said that text sets were being used in mathematics classrooms. 


Listening Centers 


Control school staff were slightly less likely than Striving Readers school staff to report 
that their school had listening centers. Specifically, at Striving Readers schools, all 
technology coordinators and LLTs and almost all (97%) LITs said that Grade 6-8 literacy 
teachers in their school had listening centers. In comparison, 89% of technology 
coordinators and 85% of LLTs in control schools reported the same. Most (94%) of the 
technology coordinators in both Striving Readers and control schools said that the 
listening centers were currently in use. Concerning the frequency of use, 89% of LLTs at 
Striving Readers schools reported that teachers use the listening centers somewhat, and 
the other 11% reported that teachers use the centers extensively. At control schools, 
responses were more varied. Approximately two thirds of LLTs (65%) reported that 
teachers use the listening centers somewhat, 24% reported that teachers use the centers 
extensively, and 12% reported that teachers do not use the centers at all. Slightly more 
than two thirds (68%) of LITs at Striving Readers schools reported that they personally 
use the centers somewhat. 


LLTs and LITs were asked how listening centers are being used, and with which 
students. In both Striving Readers and control schools, some staff reported that the 
centers are used with all students and some reported that they are used primarily for 
subgroups, including struggling readers, special education students, and ELL students. A 
few staff specified that the centers are used by small groups of students on a rotating 
basis. LLTs from control schools said that the centers are primarily used for listening to 
books and to enhance students’ comprehension and fluency. One LLT said that students 
record themselves and listen to themselves read. Responses among LLTs and LITs in 
Striving Readers schools were more varied. In addition to read-alouds and as a means to 
improve fluency and comprehension, these staff said that the centers are used to 
implement Striving Readers strategies, for AMP, to help teachers differentiate 
instruction, as a way to review science lessons (one staff said that a science teacher 
records lessons for those students who need review), and to help students who were 
absent to catch up. 


When asked how listening centers have improved instruction, LLTs and LITs from 
Striving Readers and control schools said that the centers and the technology improve 
student engagement and motivation, which they believe has an impact on the 
effectiveness of instruction. They also said that the listening centers allow students to 
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hear fluent readers. LLTs and LITs in Striving Readers schools also said that listening 
centers help them to differentiate instruction. 


Media Centers 


All staff (100%) at Striving Readers schools (technology coordinators, LLTs, and LITs) 
reported that Grade 6-8 literacy teachers in their school have media centers, and all 
technology coordinators reported that these centers are currently in use. In comparison, 
92% of technology coordinators and 79% of LLTs in control schools reported having 
media centers, and 96% of technology coordinators said that they are currently in use. 
Striving Readers staff also reported that teachers use the media centers more frequently 
than control school staff. Specifically, 100% of LLTs at Striving Readers schools 
reported that teachers use the media centers extensively, and 61% of LITs reported that 
they themselves use the media centers extensively. Of those LITs who use the media 
centers less frequently, most said that this was due to the fact that they do not have good 
materials to use in the centers, or that there is not enough time. In control schools, less 
than half (41%) of LLTs reported that teachers use the media centers extensively. 
Approximately half (47%) said that they are used somewhat, and 12% said that they are 
not used at all. One LLT who said that they do not use the media centers at all said that 
the technology is challenging, teachers don’t have enough time, and that there are too 
many students for the number of computers available. 


When LLTs and LITs in Striving Readers schools were asked how and with whom media 
centers are used, many said that the centers are used to allow students to listen to books, 
novels, or passages before, during, or after reading them directly. Several Striving 
Readers staff mentioned that students listen to readings and then ask and answer 
questions about it; that the centers are set up as literacy stations that students rotate 
through; that they help with differentiated instruction; and that they are used to improve 
fluency, intonation, and comprehension. A few staff said that they are used for AMP, to 
allow students to catch up on work that they have missed, for literacy circles or groups, 
and for PRC2 (to allow students to listen to their conversations). Staff at Striving 
Readers schools had varying responses regarding which students the media centers are 
used with/for—some said all students, others said small groups of students, and others 
specified struggling readers (particularly in the after-school program), ELL students, 
and/or special education students. Some said that the centers are used with different 
groups of students at different times of the day (e.g., with all students during classes, but 
with struggling readers during the after-school program). 


Answers given by LLTs in control schools were slightly different. Some said that 
students use the centers to do research and projects, and others said that the centers are 
used to reinforce skills and learning, prepare for tests, and measure comprehension. A 
few said that the centers are used as a means of providing support for struggling readers. 
In terms of who uses the centers, responses varied between all students and struggling 
readers. A few mentioned that teachers use the media centers to plan and prepare 
lessons, print papers, and keep records. 
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When staff were asked in what ways media centers have improved instruction, responses 
were similar among Striving Readers and control school staff. Both groups said that 
media centers help teachers to differentiate instruction for students and help to engage 
and motivate students. They also said that having media centers allows teachers to 
provide many models of fluency to students, who hear different voices reading text 
selections. A few said that media centers help to facilitate student reading of nonfiction 
for informational purposes. 


Handheld Computers 


Grade 6-8 literacy teachers in Striving Readers schools were supposed to receive 
handheld computers late in the second year of the initiative and are expected to begin 
using the handheld computers in Year 3. Because the interview data discussed in this 
section are from Year 2 of the initiative, the teachers were not yet expected to be actively 
using the handheld computers. 


In Striving Readers schools, 76% of technology coordinators and 40% of LLTs reported 
that Grade 6-8 literacy teachers have handheld computers. In addition, 76% of LITs 
reported that they and/or students in their school have handheld computers. In control 
schools, only 8% of technology coordinators and 10% of LLTs reported that literacy 
teachers have handheld computers. 


Although the majority of Striving Readers staff reported having handheld computers, 
67% of LLTs reported that teachers were not using them, and 71% of LITs reported that 
they themselves were not using them. All LLTs (100%) and most (91%) LITs also said 
that students in Striving Readers schools were not using the handheld computers. When 
these staff were asked why the handheld computers were not being used, many said that 
the school had just received the computers and that staff and/or students needed training 
on how to use them. One said that there was a problem with the software. Very few staff 
answered questions about how the handheld computers were being used or how they may 
improve instruction. Two respondents said that, at this point, the handheld computers are 
for personal use (e.g., to keep their calendar or contact information). One staff said that 
they are being used in AMP, and one said that they are being used for developing stories 
with pictures and for getting novels online. 


Integration of Technology 


Principals at Striving Readers and control schools were asked how well they believe 
technology is integrated into the literacy curriculum of their school. In both Striving 
Readers and control schools, the majority of principals (75% in Striving Readers schools 
and 79% in control schools) reported that technology is somewhat integrated. 
Approximately one fifth (21% in Striving Readers schools and 17% in control schools) 
reported that it is thoroughly integrated, and the remainder (4% in Striving Readers 
schools and 3% in control schools) said that technology is not integrated at all. 


When asked to explain their answers, Striving Readers and control school principals 


responded similarly. Many simply described the technology resources available at their 
school. As an example, one principal (each) said that every classroom in their school has 
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four to five computers, their school just got 18 laptops, or that all classrooms in their 
school have computers with Internet. In general, principals considered the level of 
technology resources at their school to be good but not great, and thought that it would be 
ideal to have even more technology available. Some principals—particularly in Striving 
Readers schools—were more specific and said that although the school has adequate 
technology, teachers need to make better use of it for instruction. They said that right 
now the students mainly use computers to do research, look at web sites, and produce 
(type and print) papers, and that only a few teachers are using technology for 
instructional purposes. Several principals said that additional professional development 
is needed to facilitate teachers using technology effectively. 


When asked to rate the impact of technology on the reading achievement of struggling 
readers in their school, principals from Striving Readers schools rated the impact higher 
than those from control schools. In particular, 70% of Striving Readers principals rated 
the impact of technology as moderate or large, but only 34% of control school principals 
rated it the same way. The majority of principals in control schools (62%) said that the 
technology had had some impact on the achievement of struggling readers. 


Other Resources in Control Schools 


Because the Striving Readers Initiative provides participating schools with resources for 
struggling readers, principals and LLTs in control schools only were asked whether their 
schools have specific literacy resources for struggling readers. Approximately three 
quarters of principals (79%) and LLTs (74%) said yes. When asked to describe these 
resources, they listed a wide range of strategies, materials, and programs. Examples 
include classroom libraries or book rooms, teacher resource rooms, leveled books, an 
online program for struggling readers, tutoring/mentoring programs, before-school and 
after-school reading programs, one-on-one coaching by a literacy teacher or 
paraprofessional, and reduced class sizes for literacy instruction. No one resource was 
cited by more than a few respondents at control schools. 


Component 7: Integrated, Progressive, and High-Quality Professional 
Development 


Higher percentages of staff from Striving Readers schools, including librarians, 
technology coordinators, and principals, reported participating in professional 
development related to Striving Readers or literacy in general either this school year or 
last summer. Specifically, 92% of librarians, 95% of technology coordinators, and all 
principals (100%) at Striving Readers schools reported participating in such professional 
development, compared to 68% of librarians, 73% of technology coordinators, and 96% 
of principals at control schools. Most principals in Striving Readers schools (89%) and 
all principals in control schools (100%) said that non-literacy staff were involved in 
professional development related to literacy. When asked to list the staff members 
involved in professional development, many principals from both Striving Readers and 
control schools said that all staff who are involved with students in Grades 6-8 
participate in professional development. They named staff positions including teachers 
of all subject areas, paraprofessionals, assistant teachers, principals, assistant principals, 
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computer/technology staff, librarians, gym teachers, art teachers, counselors, and math 
specialists. Many principals from both Striving Readers and control schools said that 
there are no staff members in their school who do not deal with literacy. 


Principals were also asked what topics had been covered in professional development 
sessions. In most cases, responses of principals from Striving Readers and control 
schools were similar. They both mentioned topics such as building and using classroom 
libraries, differentiating instruction, using small group instruction, using data to inform 
instruction, using technology, developing strategies for use with low-performing students 
or ELL students, using extended response, building vocabulary, and writing activities. 
Principals from Striving Readers schools mentioned a few topics not covered by those 
from control schools, including creating literacy stations, using handheld computers, and 
PRC2. 


The majority of staff members in both Striving Readers and control schools who 
participated in professional development said that it was very useful in providing the 
skills needed to effectively implement Striving Readers or the school’s literacy efforts. 
Three fifths (61%) of librarians, half of technology coordinators, and more than three 
quarters (76%) of principals in Striving Readers schools said that the professional 
development was very useful. In control schools, 69% of librarians, 75% of technology 
coordinators, and 70% of principals said that the professional development was very 
useful. 


When asked to explain their ratings of the usefulness of the professional development, 
Striving Readers principals had many positive things to say. In particular, they said that 
the professional development sessions provided knowledge about literacy strategies and 
techniques, allowed principals to observe in classrooms, provided the research basis 
behind the strategies used in Striving Readers, and allowed them to know what the 
expectations were of the initiative and what they should look for in their schools in terms 
of literacy instruction. A few mentioned that the networking opportunities were very 
valuable, and a few said that it was valuable to attend professional development with 
other staff from their school. A few principals commented on aspects of the professional 
development that they did not find useful. One said that they do not believe that 
principals need to know about Striving Readers in such great detail. Another felt 
similarly and did not think that principals need to observe classrooms in other schools. 


Principals in control schools felt that professional development was useful in helping 
them understand what should be covered—and how—in each grade level. They felt that 
the professional development helped them to become better instructional leaders and 
know what to look for when observing classrooms. They also felt that the professional 
development helped them to stay current on best practices and procedures that are 
currently being used in classrooms. 
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Appendix E: Definitions of Year 3 Fidelity 


Scales 


COMPONENT 1: Reading comprehension instruction for WHOLE SCHOOL, BLENDED INTERVENTION 


Sub-Components 


Individual/Summary Items 


Scores 


Sub-Component 1: 
Individual and small 
group instruction 


LIST Survey - from ELA Teachers only 


Q7. How often do you use the following grouping 
structures in your classes? 


° Individual Work 
° Small groups or Pairs 


Never=0; Less than once a month=2; 1- 
3 times a month=4; 1-3 times a week=6; 
4-5 times a week=8; Multiple times a 
day=10 


Score 


Sum of the Above Items/2 (Range 
from 0 to 10) 


Q18. For each of the materials listed below, please 
indicate which grouping strategies are supported by 
your use of that material in your classroom. 


e Listening centers 


e Media Centers 


e Classroom Library 


For each item: 


=0 if small group AND individual work 
are NOT checked 

=5 if either small group OR individual 
work are checked 


e Vocabulary Notebooks 


e _ Reading response notebooks 


=10if BOTH small group AND 
individual work are checked 


Score 


Sum of the Above Items/5 (Range 
from 0 to 10) 


Total Sub-Component 1 Score (Individual and Small 
Group Instruction) 


Sum of the Above Scores/2 (Range 
from 0 to 10) 


Sub-Component 2: Use 
of gradual release 
model to provide 
direct, explicit 
instruction and 
scaffold learning for 
students. 


Q3. How often do you use the following practices to 
help students increase reading comprehension? 


LIST Survey - from ELA Teachers only 


e Use of the gradual release of responsibility 
model 


Q4. How often do you use the following practices to 
help students build their vocabulary knowledge? 


e Use of the gradual release of responsibility 
model 


Never=0; Less than once a month=2.5; 
1-3 times a month=5; 1-3 times a 
week=7.5; 4-5 times a week=10; 


Total Sub-Component 2 Score (Gradual Release 
Model Setting) 


Sum of the Above items/2 (Range 
from 0 to 10) 


Sub-Component 3: 
Instruction anchor for 
all classrooms and 
content areas is focused 
on comprehension. 


LIST Survey - from ELA Teachers only 


Q3. How often do you use the following practices to 
help students increase reading comprehension? 


Explicit instruction in use of... 


For each item, give the following 
values: 

Never=0; Less than once a month=1; 1- 
3 times a month=2; 1-3 times a week=3; 


0=0; 1=2.5; 2=5; 3=7.5; and 4 through 


- _..,summarizing 
oo tee MeSOMEE 4-5 times a week=4 
; ...Predicting 
-  .,.Text structure Then sum all items and recode as 
- _ ,,.Visualization following: 
- ,..Inferri 
nferring _ highest=10 
. ...Metacognition 
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COMPONENT 1: Reading comprehension instruction for WHOLE SCHOOL, BLENDED INTERVENTION 


Sub-Components 


Individual/Summary Items 


Scores 


Score 


Range from 0 to 10 


Q3a. How often do you use the following practices to 
help students increase reading comprehension? 


- Establishing the purpose for reading. 


- Monitoring students’ comprehension through 
questioning. 


- Use of before, during, and after (BDA) reading 
strategies for comprehension instruction 


Never=0; Less than once a month=2.5; 
1-3 times a month=5; 1-3 times a 
week=7.5; 4-5 times a week=10; 


Q3b. How often do you use the following practices to 
help students increase reading comprehension? 


— Making connections to background knowledge. 


— Making connections between texts. 


—  Synthesizing information within text or across 
texts. 


Never=0; Less than once a month=3.3; 
1-3 times a month=6.7; 1-3 times a 
week=10; 4-5 times a week=10; 


Score 


Sum of the Above Items/6 (Range 
from 0 to 10) 


Q6. How often do you use the following techniques to 
help struggling readers develop better reading 
strategies and skills? 


e Everybody Reads To (ERT) 


° Exclusion Brainstorming 

e List-Group-Label 

° Interactive Notation System for Effective 
Reading and Thinking (INSERT) 

° Guided Reading and Summarizing 
Procedure (GRASP) 


Never=0; Less than once a month=3.3; 
1-3 times a month=6.7; 1-3 times a 
week=10; 4-5 times a week=10; 


Q6. How often do you use the following techniques to 
help struggling readers develop better reading 
strategies and skills? 


e ReQuest 


e Predict-Locate-Add-Note (PLAN) 
e ABC Graffiti 


Never/ Not Familiar=0; Less than once 
a month=5; 1-3 times a month=10; 1-3 
times a week=10; 4-5 times a week=10 


Score 


Sum of the Above Items/8 (Range 
from 0 to 10) 


Q16. For each of the materials listed across the top of 
the chart below, please indicate which literacy 
instructional goals are supported by your use of that 
material in your classroom: Reading Comprehension 


e _ Listening centers 


e Media centers 


e Classroom library 


Not checked = 0; Checked=10 


Score 


Sum of the Above Items/3 (Range 
from 0 to 10) 


Total Sub-Component 3 Score (Systematic 
Comprehension) 


Sum of the Above Scores/4 (Range 
from 0 to 10) 


Sub-Component 4: 
PRC2 instructional 
framework, text sets, 


LIST Survey - from ELA Teachers only 


Q3. How often do you use the following practices to 
help students increase reading comprehension? 


Never=0; Less than once a month=3.3; 
1-3 times a month=6.7; 1-3 times a 
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COMPONENT 1: Reading comprehension instruction for WHOLE SCHOOL, BLENDED INTERVENTION 


Sub-Components 


Individual/Summary Items 


Scores 


and technology are 
used fluidly and 
alternately to support 
differentiated 
instruction and 
increase student 
motivation, 
engagement, and 
understanding. 


e PRC2 


Q4. How often do you use the following practices to 
help students build their vocabulary knowledge? 


e PRC2 


Q3. How often do you use the following practices to 
help students increase reading comprehension? 


e Using differentiated instruction 


week=10; 4-5 times a week=10; 


Q8. Considering your own instruction, how often do you 
apply differentiated instruction in your classroom? 


Never=0; Rarely=2.5; Occasionally=5; 
About half the time=7.5; Most of the 
time=10; Almost every lesson or 
activity=10 


Q17 Use of Listening Centers for differentiating 
instruction for struggling readers 


Q17 Use of Media Centers for differentiating instruction 
for struggling readers 


struggling readers 


Q24 Use of Palm Pilots for differentiating instruction for 


Not checked = 0; Checked=10 


Total Sub-Component 4 Score (Use of PRC2, Text 
Sets and Technology for Differentiated Instruction) 


Sum of the Above Items/7 (Range 
from 0 to 10) 


Sub-Component 5: 
Systematic approach to 
teaching academic 
content vocabulary in 
all subjects using 
Robert Marzano’s 
Building Academic 
Content Vocabulary 


Q4. How often do you use the following practices to 
help students build their vocabulary knowledge? 


LIST Survey - from ELA Teachers only 


Explicit instruction in vocabulary 


Modeling the use of word parts 


Review of vocabulary words 


Use of vocabulary notebooks 


Use of before, during, and after (BDA) 
reading strategies for vocabulary instruction 


Never=0; Less than once a month=3.3; 
1-3 times a month=6.7; 1-3 times a 
week=10; 4-5 times a week=10; 


Score 


Sum of the Above Items/5 (Range 
from 0 to 10) 


Q16. For each of the materials listed across the top of 
the chart below, please indicate which literacy 
instructional goals are supported by your use of that 
material in your classroom: Vocabulary 
Development 


e __ Listening centers 


e Media centers 


e Classroom library 


e Vocabulary notebooks 


Checked =0; Not Checked =10. 


Score 


Sum of the Above Items/4 (Range 
from 0 to 10) 


Total Sub-Component 5 Score (Vocabulary) 


Sum of the Above Scores/2 (Range 
from 0 to 10) 


Total Component | - Blended Intervention Score: 


Sum of the Above Sub-Components/5 
(Range from 0 to 10) 
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COMPONENT 2: 


3 students 


Reading comprehension instruction for TARGETED intervention model for Tier 2 and TIER 


Sub-Components 


Sub-Component 
6: Teachers and 
Literacy 
Intervention 
Teachers 
collaboration in 
instructional 
planning and 
progress 
monitoring. 


Individual/Summary Items 


LIST Survey — from LIT only (about targete 


Scores 


d intervention) 


Q6. How often do you meet with ELA classroom teachers at the 
following grade levels to discuss instruction-related issues regarding 
your work with students in the Targeted intervention group. 


e Grade 6 teachers 


e Grade 7 teachers 


° Grade 8 teachers 


For Q6a, 6b, and 6c: 

Never=0; Less than once a 
month=3.3; 1-3 times a month=6.7; 
1-3 times a week=10; 4-5 times a 
week=10 


Score 


Sum of the Above Items/3 (Range 
from 0 to 10) 


Q7. In which setting(s) do you meet or collaborate with ELA 
classroom teachers? (Check all that apply for each grade) 


e Grade 6 teachers 


e Grade 7 teachers 


e Grade 8 teachers 


Score 


For Q7a, 7b, and 7c: 

Sum of the following (for a total 
score of 10): Scheduled one-on-one 
meetings = 3.3 or 0 (if not checked); 
Grade-level team meetings = 3.3 (if 
checked) or 0 (if not checked); 
Literacy leadership team meetings 
=3.3 (if checked) or 0 (if not 
checked) 


Sum of the Above Items/3 (Range 
from 0 to 10) 


Q8. How often do you meet with SIXTH-GRADE classroom 
teachers to discuss implementing each of the following instructional 
methods for students in the in-class Targeted Intervention group 
(Tiers 2-3)? 


e Differentiated instruction 
e Student groupings 


e ~—_— Use of Striving Readers texts sets, text set teacher guides, 
technology, classroom library, school library 


e Use of specific Striving Readers comprehension strategies 
for reading 


e _ Using specific Striving Readers instructional techniques 
for comprehension instruction 


e Using specific Striving Readers instructional techniques 
for vocabulary instruction 


e Using specific Striving Readers instructional techniques 
for fluency instruction 


e _ Discussing specific students’ reading progress 


e Coordinating instruction between lessons for the whole 
class and lessons for the Targeted Intervention group 


e —_ Using student assessment data for instructional planning 


Never=0; Less than once a month=5; 
1-3 times a month=10; 1-3 times a 
week=10; 4-5 times a week=10 


Score 


Sum of the Above Items/10 (Range 
from 0 to 10) 
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COMPONENT 2: 
3 students 


Reading comprehension instruction for TARGETED intervention model for Tier 2 and TIER 


Sub-Components 


Individual/Summary Items 


Scores 


LIST Survey — from ELA Teachers 


Q38. How often do you meet or collaborate with the LIT in the 
following settings? 


a) 


Scheduled one-on-one meetings 


c) 


Grade-level team meetings 


Score 


Q39. To what extent has your collaboration with the LIT facilitated 
your efforts to use the following methods to support struggling 
readers in your class? 


For 6th grade teachers: Never=0; 
Less than once a month=3.3; 1-3 
times a month=6.7; 1-3 times a 
week=10; 4-5 times a week=10 


For 7th or 8th grade teachers: 
Never=0; Less than once a month=5; 
1-3 times a month=10; 1-3 times a 
week=10; 4-5 times a week=10 


Sum of the Above Items/2 (Range 
from 0 to 10) 


e _ Differentiating instruction 


e Scaffolding of instruction 


e Student groupings 


Not at all=0; To a small extent=5; To 
a moderate extent=10; To a large 


Use of the Whole-Part-Whole instructional model 


Using the media center 
Using listening centers 


Using assessment data to monitor student progress 


Using student assessment data for instructional planning 


extent=10. 


(Also 0 if Q38a and 38c=0) 


Score 


Q40. To what extent has your collaboration with the LIT facilitated 
your ability to provide effective instruction in the following areas for 
struggling readers? 


Sum of the Above Items/8 (Range 
from 0 to 10) 


e Comprehension 


e Fluency 


e Vocabulary 


Writing skills 


Word parts 
Word recognition 


Spelling 


e _ Reading/literacy in content areas 
Score 


Not at all=0; To a small extent=5; To 
a moderate extent=10; To a large 
extent=10. 


(Also 0 if Q38a and 38c=0) 


Sum of the Above Items/8 (Range 
from 0 to 10) 


Sub-Component 


Total Sub-Component 6 Score (Collaboration) 


LIST Survey - from LIT only (section about tar, 


Sum of the Above Scores/6 (Range 
from 0 to 10) 


eted intervention) 


7: Explicit 
instruction in 


QI! How often do you use the following grouping structures during 
the push-in intervention with Tier 2and 3 students? 


small group 
setting for Tier 2- 
3 students for 


e Small groups/Pairs 


Score 


Never=0; Less than once a 
month=2.5;1-3 times a month=5;1-3 
times a week=7.5;4-5 times a 
week=10; Multiple times a day=10 


Range from 0 to 10 
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COMPONENT 2: 
3 students 


Reading comprehension instruction for TARGETED intervention model for Tier 2 and TIER 


Sub-Components 


approximately 20- 
30 minutes per 
day, in 7 core 
comprehension 
strategies: 
summarization, 
predicting, 
inferring, 
metacognition, 
visualization, 
questioning, and 
text structure. 
[Also addresses 
grouping, 
vocabulary 
instruction, 
fluency 


Individual/Summary Items 

Q2. During your work in the regular classroom with students in 
the Targeted intervention group (Tier 2 and 3 students), how often 
do you use the following practices to help struggling readers increase 
reading comprehension? 


e Explicit instruction in use of summarizing 
e Explicit instruction in use of questioning 


e — Explicit instruction in use of predicting 


e — Explicit instruction in using text structure 


e __ Explicit instruction in use of visualization 


e Explicit instruction in use of inferring 


e _ Explicit instruction in use of metacognition 


Scores 


For each item, give the following 
values: 

Never=0; Less than once a month=1; 
1-3 times a month=2; 1-3 times a 
week=3; 4-5 times a week=4 


Then sum all items and recode as 
following: 

0=0; 1=2.5; 2=5; 3=7.5; and 4 
through highest=10 


Score 

Q2. During your work in the regular classroom with students in 
the Targeted intervention group (Tier 2 and 3 students), how often 
do you use the following practices to help struggling readers increase 
reading comprehension? 


Range from 0 to 10 


e Establishing the purpose for reading 


e Monitoring students’ comprehension through questioning 


e Making connections to background knowledge 


e Making connections between texts 


e Synthesizing information within text or across texts 


e _ Using differentiated instruction 


e UseofBDA reading strategies for comprehension 
instruction 


e ~—-—d Use of the gradual release of responsibility model for 
reading comprehension instruction 


Never=0; Less than once a 
month=2.5; 1-3 times a month=5; 1-3 
times a week=7.5; 4-5 times a 
week=10; 


Q2. During your work in the regular classroom with students in 
the Targeted intervention group (Tier 2 and 3 students), how often 
do you use the following practices to help struggling readers increase 
reading comprehension? 


= Using PRC2 for comprehension instruction 


Never=0; Less than once a 
month=3.3; 1-3 times a month=6.7; 
1-3 times a week=10; 4-5 times a 
week=10; 


Score 


Sum of the Above Items/9 (Range 
from 0 to 10) 


Q3. During your work in the regular classroom with students in 
the Targeted intervention group (Tier 2 and 3 students), how often 
do you use the following practices to help struggling readers build 
their vocabulary knowledge? 


Never=0; Less than once a 


e —_ Explicit instruction in vocabulary 


e Modeling the use of word parts 


e Use of the PRC2 for vocabulary development 


e ~—_- Use of the gradual release of responsibility model for 
vocabulary instruction 


month=3.3; 1-3 times a month=6.7; 
1-3 times a week=10; 4-5 times a 
week=10; 
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COMPONENT 2: 
3 students 


Reading comprehension instruction for TARGETED intervention model for Tier 2 and TIER 


Sub-Components 


Individual/Summary Items 

Q3. During your work in the regular classroom with students in 
the Targeted intervention group (Tier 2 and 3 students), how often 
do you use the following practices to help struggling readers build 
their vocabulary knowledge? 


Scores 


Never =0; Less than once a 
month=2.5; 1-3 times a month=5; 1-3 


e —_ Review of vocabulary words 
e Use of vocabulary notebooks 


times a week=7.5; 4-5 times a 
week=10 


e Use of BDA reading strategies for vocabulary instruction 


Score 


Sum of the Above Items/7 (Range 
from 0 to 10) 


Q4. During your work in the regular classroom with students in 
the Targeted intervention group (Tier 2 and 3 students), how often 
do you use the following practices to help struggling readers develop 
fluency? 


e —_ Teacher interactive read aloud 


Never =0; Less than once a 
month=3.3; 1-3 times a month=6.7; 
1-3 times a week=10; 4-5 times a 


e Shared reading 


week=10 


e — Explicit instruction in guided oral reading 


e Students listen to audio books, play aways 
Q4. During your work in the regular classroom with students in 
the Targeted intervention group (Tier 2 and 3 students), how often 
do you use the following practices to help struggling readers develop 
fluency? 


Never =0; Less than once a 
month=2.5; 1-3 times a month=5; 1-3 
times a week=7.5; 4-5 times a 


e Teacher read aloud 


week=10 


e Modeling reading for students 


Score 


Sum of the Above Items/6 (Range 
from 0 to 10) 


Q5 How often do you use the following techniques to help 
struggling readers develop better reading strategies and skills? 


Never =0; Less than once a month=5; 


e ~=©ReQuest 


1-3 times a month=10; 1-3 times a 


e =6©ABC Graffiti 


week=10; 4-5 times a week=10 


e —_ Predict-Locate-Add-Note (PLAN) 


Q5 How often do you use the following techniques to help 
struggling readers develop better reading strategies and skills? 


e _ Everybody Reads To (ERT) 


Never =0; Less than once a 


e __ Exclusion Brainstorming 


month=3.3; 1-3 times a month=6.7; 


e —__List-Group-Label 


1-3 times a week=10; 4-5 times a 


e Interactive Notation System for Effective Reading and 
Thinking (INSERT) 


week=10 


e Guided Reading and Summarizing Procedure (GRASP) 


Score 


Sum of the Above Items/8 (Range 
from 0 to 10) 


Total Sub-Component 7 Score (Comprehension) 


Sum of the Above Scores/6 (Range 
from 0 to 10) 


Total Component 2 - Targeted Intervention Score: 


(Sum of above Sub-component 
Scores)/2 = Range from 0 to 10 
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COMPONENT 3: Reading comprehension instruction for intensive intervention model for Tier 3 students 


Sub-Components 
Sub-Component 8: 
Increased time—an 
additional 240 minutes 
of direct and supported 
instruction beyond the 
intervention that occurs 
during the regular 
school day. 


Sub-Component 9: 
Small groups setting: 15 
to 1 teacher student 
ratio. 


Sub-Component 10: 
Explicit and systematic 
instruction in seven core 
comprehension 
strategies: 
summarization, 
predicting, inferring, 
metacognition, 
visualization, 
questioning, and text 
structure (strategies 
introduced one at a 
time) during the 
additional 240 minutes of 
supported instruction. 


Sub-Component 11: 
Teaching of high volume 
and depth of academic 
vocabulary. 


Individual/Summary Items 


Scores 


AMP Schedule and Attendance Records 


Total # minutes AMP Classes should meet per week (240) * 
Number of Weeks (26) = 6240 minutes 


Score = (Total Number of Minutes the Program 
Operated at Each School)/624, max=10 


Average number of minutes attended by student by school. 


Score = (Total Number of Minutes Attending 
Averaged Across Students)/624, max=10 


Total Sub-Component 8 Score (Increased Time) 


Sum of above Items/2 (Range from 0 to 10) 


AMP Enrollment Records 


Number of Students and Teachers Per Class 


(# Teachers)/(# Students) X 150, max=10 


Total Sub-Component 9 Score (Small Group Settin 


) 
LIST Survey - from LIT and AMP teachers only 


Q15. How often do you use the following practices or 
materials with Tier 3 students in the AMP after-school 
program to help them increase reading comprehension? 


e __ Explicit instruction in use of summarizing 


Explicit instruction in use of questioning 


Explicit instruction in use of predicting 


Explicit instruction in using text structure 


Explicit instruction in use of visualization 


Explicit instruction in use of inferring 


e Explicit instruction in use of metacognition 


Range from 0 to 10 


(section about intensive intervention) 


For each item, give the following values: 
Never=0; Less than once a month=1; 1-3 times 
a month=2; 1-3 times a week=3; 4-5 times a 
week=4 


Then sum all items and recode as following: 
0=0; 1=2.5; 2=5; 3=7.5; and 4 through 
highest=10 


Total Sub-Component 10 Score (Systematic 
Comprehension) 


Q16. How often do you use the following practices or 

materials with Tier 3 students in the AMP afterschool 

program to help them build their vocabulary knowledge? 
e Explicit instruction in vocabulary 


e Modeling the use of word parts 


e —_ Use of the gradual release of responsibility model 


for vocabulary 


Range from 0 to 10 


LIST Survey - from LIT and afterschool teachers only (section about intensive intervention) 


Never=0; Less than once a month=3.3; 1-3 
times a month=6.7; 1-3 times a week=10; 4-5 
times a week=10 


Q16. How often do you use the following practices or 
materials with Tier 3 students in the AMP afterschool 
program to help them build their vocabulary knowledge? 
e Review of vocabulary words 
e _Use of vocabulary notebooks 


Use of before, during, and after (BDA) reading 
strategies for vocabulary instruction 


Never=0; Less than once a month=2.5; 1-3 
times a month=5; 1-3 times a week=7.5; 4-5 
times a week=10; 
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COMPONENT 3: Reading comprehension instruction for intensive intervention model for Tier 3 students 


Sub-Components 


Sub-Component 12: 
Guided fluency practice. 


Individual/Summary Items 


Total Sub-Component 11 Score (Vocabulary) 


Scores 
Sum of the Above Items/6 (Range from 0 to 
10) 


LIST Survey - from LIT and afterschool teachers only (section about intensive intervention) 


Q17. How often do you use the following practices or 
materials with Tier 3 students in the AMP afterschool 
program to help them develop fluency? 


e —_ Teacher interactive read aloud 


e Shared reading 


e Students listen to audio books, play aways 


e = Use of the gradual release of responsibility model 
for fluency instruction 


Never =0; Less than once a month=3.3; 1-3 
times a month=6.7; 1-3 times a week=10; 4-5 
times a week=10 


Q17. How often do you use the following practices or 
materials with Tier 3 students in the AMP afterschool 
program to help them develop fluency? 


e _ Teacher read aloud 


e Modeling reading for students 


Never =0; Less than once a month=2.5; 1-3 
times a month=5; 1-3 times a week=7.5; 4-5 
times a week=10 


Total Sub-Component 12 Score (Guided fluency 
practice) 


Total Component 3 - Intensive Intervention Score: 


Sum of the Above Items/6 (Range from 0 to 
10) 

(Sum of above Sub-component Scores)/5 = 
Range from 0 to 10 
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COMPONENT 4: FREQUENT, PURPOSEFUL ASSESSMENT AND ADJUSTMENT OF INSTRUCTION WITH 
SCREENING, DIAGNOSTIC, AND PROGRESS-MONITORING TOOLS AND DATA-DRIVEN INSTRUCTION 
STRUCTURED THROUGH A TEAM-BASED SYSTEM OF LEADERSHIP AND SUPPORT 


Sub-Components 
Sub-Component 13: 
Purposeful assessments with 
screening, diagnostic, and 
progress-monitoring tools 
and data-driven instruction 
structured through a team- 
based system of leadership 
and support FOR WHOLE 
SCHOOL BLENDED 
INTERVENTION 


Individual/Summary Items 
Principal Interviews 


Scores 


Q4. To what extent is student assessment data being used for 
this purpose? 


e Screening students’ ability levels for placement in 
intervention programs 


e Diagnosing students’ strengths and support needs for 
placement in specific courses or instructional groups 


e Identifying trends in fluency and comprehension 
abilities across groups of students 


e Identifying trends in vocabulary knowledge across 
groups of students 


e Monitoring overall student progress for the purpose of 
assessing success of instructional programs and methods 


e __ Differentiating instruction 


e Planning on-site professional development 


Not at all=0; To s Small 
Extent=3.3; To a Moderate 
Extent=6,7; To a Large Extent=10 


Score 


Sum of above Items/7 (Range 
from 0 to 10) 


Q2c. Overall, rate the quality of the literacy team’s 
performance in the following areas 
e _Using assessment data and or student work to plan 
instruction 


[If school does not have a literacy 
team=0] Poor=0; Fair=3.3; 
Good=6.7; Excellent=10; Not 
Sure=0 


Score 


Range from 0 to 10 


Q3c. Overall, rate the quality of the grade level team’s 
performance in the following areas 


e __Using assessment data to plan instruction 


e Using assessment data to establish vertical and 
horizontal literacy goals by grade level 


[If school does not have a grade- 
level team=missing] Poor=0; 
Fair=3.3; Good=6.7; Excellent=10; 
Not Sure=0 


Score 


Sum of above Items/2 (Range 
from 0 to 10) 


LIST Surveys — from ELA Teachers 


Q9 Indicate how you use the data from the following 
assessments: 


[From Not used=0 to Used in all 
intended ways=10. No extra points 
for additional applications. ] 


Reading Benchmark Assessment 


Not using=0; Screening = +2.5; 
Benchmarking=+2.5; Progress 
monitoring= +2.5 Assess 
outcomes=+2.5 


ISAT 


Not Using=0; Screening=+5; 
Outcome=+5 
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COMPONENT 4: FREQUENT, PURPOSEFUL ASSESSMENT AND ADJUSTMENT OF INSTRUCTION WITH 
SCREENING, DIAGNOSTIC, AND PROGRESS-MONITORING TOOLS AND DATA-DRIVEN INSTRUCTION 


STRUCTURED THROUGH A TEAM-BASED SYSTEM OF LEADERSHIP AND SUPPORT 


Sub-Components 


Sub-Component 14: Data- 
driven instruction FOR 
INTENSIVE 
INTERVENTION 


Individual/Summary Items 


Scores 


Informal assessments 


Not using=0; Diagnostic = +2.5; 
Benchmarking=+2.5; Progress 
monitoring= +2.5 Assess 
outcomes=+2.5 


Fluency Snapshots 


Not Using=0; Screening=+10 


Spelling inventories 


Not using=0; Screening=+2; 
Diagnostic = +2; 
Benchmarking=+2; Progress 
monitoring= +2 Assess 
outcomes=+2 


Score 


Q10. Indicate extent you use student assessment data for 
each of the following purposes: 


a) Placing students in intervention programs; 


b) Differentiating instruction; 
c) Identifying skills that need to be re-taught; 


d) Monitoring student reading progress; 


e) Creating instructional groups 


Sum of above Items/5 (Range 
from 0 to 10) 


Not at all=0; To a small extent=3.3; 
To a moderate extent=6.7; To a 
large extent=10 


Score 


Sum of above Items/5 (Range 
from 0 to 10) 


Q11-12 Do you currently have grade-level teams at your 
school? If YES: Overall, rate the grade-level team’s ability 
to use classroom assessment data in the following ways: 


Address the literacy needs of all students 


Address the needs of struggling readers; 


Formalize lesson plans; 


Identify students who are eligible for targeted 
interventions; 


Identify strengths; 


Identify teaching and learning strategies 


Improve classroom practice 


No grade-level team= missing on 
all items; or Poor=0; Fair=3.3; 
Good=6.7; Excellent=10; Not 
sure=0 


Score 


Sum of above Items/7 (Range 
from 0 to 10) 


Total Sub-Component 13 Score 


Sum of the Above Scores/6 
(Range from 0 to 10) 


LIST Surveys — from LIT Teacher and AMP Teachers (about the intensive intervention) 


Q19. Please indicate the extent to which you use student 
assessment data for each of the following purposes within the 
afterschool program: 


e __ Differentiating instruction 


e _ Identifying skills that need to be retaught 


Not at all=0; To Some Extent=3.3; 
To a Moderate Extent=6.7; To a 
Large Extent=10 


COMPONENT 4: FREQUENT, PURPOSEFUL ASSESSMENT AND ADJUSTMENT OF INSTRUCTION WITH 
SCREENING, DIAGNOSTIC, AND PROGRESS-MONITORING TOOLS AND DATA-DRIVEN INSTRUCTION 
STRUCTURED THROUGH A TEAM-BASED SYSTEM OF LEADERSHIP AND SUPPORT 

Sub-Components Individual/Summary Items Scores 


e Monitoring student reading progress 


e _Creating instructional groups 


Sum of the Above Items/4 (Range 
Total Sub-component 14 Score from 0 to 10) 
Total Component 4 - Assessment Score: 


(Sum of above Sub-component 
Scores)/2 = Range from 0 to 10 


E-12 


COMPONENT 5: HIGH QUALITY, HIGH INTEREST MATERIALS - Highly motivating reading materials integrated with 


engaging technolog 
Components 


and audio resources. 


Individual/Summary Items 


Scores 


Sub-Component 15: Text 
Sets 

(NOTE: Not included in 
the calculation of 
Component 5 Score) 


Principal Interviews 


Q8. Are the school-wide text sets being used in the 
content area classrooms? 


e Social Studies 


No or Not Used=0; Used=10; Don’t 


e Science 
e Mathematics 


Know=Missing 


Total Sub-Component 15 Score (Text Sets) 


Sub-Component 16: 
School Library 


Sum of Above Scores/3 (Range from 0 to 10) 


LIST Surveys — From ELA Teachers 


Q15a. For each of the materials listed below, indicate 
how frequently you currently use the materials to 
teach literacy: School Library 


N/A (Do Not Have)=0; Not Currently Using=0; 
Less than once a month=2.5; 1 to 3 times a 
month=5; | to 3 times a week=7.5; 4 to 5 times 
a week=10 


Score 


Range from 0 to 10 


Q15b Rate you comfort level: School Library 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 
4=7.5; 5 Very comfortable=10 


Score 


Range from 0 to 10 


Q31. How often do you take your class to the library? 


Never=0; Rarely=2.5; Sometimes=5; 
Often=7.5; Almost daily or daily=10 


Score 


Range from 0 to 10 


Q33. How does the librarian work with you? 


a) Does not work with me=0; If checked b) or 
c)=5; If checked b) and c)=10 
(Also, if no librarian, then =0) 


Score 


Range from 0 to 10 


Q34. To what extent does the librarian consult with 
classroom teachers in using Striving Readers library 
funds to order reading materials that are grade level 
and content appropriate? 


Not at all=0; To a small extent=3.3; To a 
moderate extent=6.7; To a large extent=10; 
Don’t know=0 


Score 


Range from 0 to 10 


Q35. To what extent does the librarian consider 
students’ needs and reading abilities when ordering 
books and other reading material with Striving 
Readers library funds? 


Not at all=0; To a small extent=3.3; To a 
moderate extent=6.7; To a large extent=10; 
Don’t know=missing 


Score 


Range from 0 to 10 


Q36. To what extent does the librarian consider 
students’ interests and motivation when ordering 
books and other reading material with Striving 
Readers library funds? 


Not at all=0; To a small extent=3.3; To a 
moderate extent=6.7; To a large extent=10; 
Don’t know=missing 


Score 


Range from 0 to 10 


Q37. How does the librarian work with your 
students? (Check all that apply.) 


e Does not work with my students. 


e Works with students on research skills. 
e _ Directs students to resources tied to 
curriculum. 


e Conducts read-alouds. 


“Does not work with my students” or "no 
librarian"= 0 Otherwise: give 2.5 points per 
check with a maximum of 10 points 
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COMPONENT 5: HIGH QUALITY, HIGH INTEREST MATERIALS - Highly motivating reading materials integrated with 
engaging technology and audio resources. 


Components 


Individual/Summary Items 


Scores 


e —_ Provides students with information about 
extracurricular academic activities (e.g., spelling bee, 
writing competitions, events). 


e___ Assists students with class projects. 


e Teaches students how to navigate Internet 
resources. 


e Guides struggling readers to summer 
programs. 


Score 


Range from 0 to 10 


LIST Surveys — From LITs 


Q7a. For each of the materials listed below, indicate 
how frequently you currently use the materials to teach 
literacy: 


N/A (Do Not Have)=0; Not Currently 
Using=0; Less than once a month=2.5; 1 to 3 
times a month=5; | to 3 times a week=7.5; 4 


° School Library 


to 5 times a week=10 


Q7b Rate you comfort level: 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 


° School Library 


4=7.5; 5 Very comfortable=10 


Score 


Sum of Above Items/2 (Range from 0 to 
10) 


Total Sub-Component 16 Score (School Library) 


Sum of Above Scores/9 (Range from 0 to 
10) 


Sub-Component 17: 
Classroom Library 


LIST Surveys — From ELA Teachers 


Q15a. For each of the materials listed below, indicate 
how frequently you currently use the materials to teach 
literacy: Classroom Library 


N/A (Do Not Have)=0; Not Currently 
Using=0; Less than once a month=2.5; | to 3 
times a month=5; 1 to 3 times a week=7.5; 4 to 
5 times a week=10 


Score 


Range from 0 to 10 


Q15b Rate you comfort level: Classroom Library 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 
4=7.5; 5 Very comfortable=10 


Score 


Range from 0 to 10 


Q16. For each of the materials listed, please indicate 
which literacy instructional goals are supported by 
your use of that material in your classroom: 
Classroom Library 


e Vocabulary development 


e Fluency 


e Reading Comprehension 


e Word Recognition 


e To develop students’ self-directed learning 


e _To-supplement students’ textbook reading 


e _To activate students’ prior knowledge 


Not Using = 0; Checked=10 


Score 


Sum of Above Items/7 (Range from 0 to 10) 


Q27. Please check the ways that you use your 
classroom libraries: 


e _ For content area instruction 


e __ For independent reading 


e For small group instruction 


e —_- For read alouds 


For each item: Not checked=0; Checked=10 
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COMPONENT 5: HIGH QUALITY, HIGH INTEREST MATERIALS - Highly motivating reading materials integrated with 
engaging technology and audio resources. 


Components 


Individual/Summary Items 


Scores 


Score 


Sum of Above Items/4 (Range from 0 to 10) 


Q28. Do you use interest inventories to help 
students self select reading materials? 


No=0; Yes=10 


Score 


Range from 0 to 10 


Q29. Do you use interest inventories to guide your 
purchases for the classroom library? 


No=0; Yes=10 


Score 


Range from 0 to 10 


Q30 My classroom library... 

Is easily accessible to students 

Is well organized and in good shape 
Has a checkout system in place 


Includes a variety of reading materials 


Not at all true=0; Slightly true=3.3; Somewhat 


Includes a variety of texts 


Has reading materials grouped by genre 
Has reading materials clearly labeled 


Has both non-fiction and fiction books 


true=6.7; Very true=10 


Score 


Sum of the Above Items/8 (Range from 0 to 
10) 


LIST Surveys — From LITs 


Q7a. For each of the materials listed below, indicate 
how frequently you currently use the materials to teach 
literacy: 


° Classroom Library 


N/A (Do Not Have)=0; Not Currently 
Using=0; Less than once a month=2.5; 1 to 3 
times a month=5; | to 3 times a week=7.5; 4 to 
5 times a week=10 


Q7b Rate you comfort level: 
Classroom Library 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 
4=7.5; 5 Very comfortable=10 


Score 


Sum of Above Items/2 (Range from 0 to 10) 


Total Sub-Component 17 Score (Classroom Libraries) 


Sum of the Above Scores/8 (Range from 0 to 
10) 


Sub-Component 18: Other 
Non-Technology 
Resources/Materials 


LIST Surveys — From 


ELA Teachers 


Q15a. For each of the materials listed below, indicate 
how frequently you currently use the materials to teach 
literacy: 


N/A (Do Not Have)=0; Not Currently 
Using=0; Less than once a month=2.5; 1 to 3 
times a month=5; 1 to 3 times a week=7.5; 4 to 


e Vocabulary Notebooks 5 times a week=10 
Score Range from 0 to 10 
N/A (Do Not Have)=0; Not Currently 
Using=0; Less than once a month=2.5; | to 3 
times a month=5; 1 to 3 times a week=7.5; 4 to 
e _ Reading response notebooks 5 times a week=10 
Score Range from 0 to 10 
N/A (Do Not Have)=0; Not Currently 
Using=0; Less than once a month=2.5; 1 to 3 
times a month=5; | to 3 times a week=7.5; 4 to 
e Other informational text sets 5 times a week=10 
Score Range from 0 to 10 


Q15b Rate you comfort level: 


Vocabulary Notebooks 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 
4=7.5; 5 Very comfortable=10 


Score 


Range from 0 to 10 


COMPONENT 5: HIGH QUALITY, HIGH INTEREST MATERIALS - Highly motivating reading materials integrated with 
engaging technology and audio resources. 


Components 


Individual/Summary Items 


Scores 


e __Reading response notebooks 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 
4=7.5; 5 Very comfortable=10 


Score Range from 0 to 10 
N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 
e Other informational text sets 4=7.5; 5 Very comfortable=10 
Score Range from 0 to 10 


Q16. For each of the materials listed, please indicate 
which literacy instructional goals are supported by 
your use of that material in your classroom: 
Vocabulary Notebooks 


e Vocabulary development 
e _Word parts 

e Spelling 

Grammar 

To develop students’ self-directed learning 

To activate students’ prior knowledge 


Not Using = 0; Checked=10 


Score 


Sum of Above Items/7 (Range from 0 to 10) 


LIST Surveys — From LITs 


Q7a. For each of the materials listed below, indicate 
how frequently you currently use the materials to teach 
literacy: 


N/A (Do Not Have)=0; Not Currently 
Using=0; Less than once a month=2.5; 1 to 3 
times a month=5; | to 3 times a week=7.5; 4 to 


e Vocabulary Notebooks 5 times a week=10 
Q7b Rate you comfort level: N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 
e Vocabulary Notebooks 4=7.5; 5 Very comfortable=10 


Score 


Sum of Above Items/2 (Range from 0 to 10) 


Total Sub-component 18 Score (Other Non-Technology Materials) 


Sum of the Above Scores/8 (Range from 0 to 
10) 


Sub-Component 19: Use of 
HHC for reading 
instruction as part of the 
targeted intervention 
model 


LIST Survey - from LIT only (Section about targeted intervention) 


Q10. How often do students use handheld computers 
(Palm Pilots) during Targeted Intervention instruction? 


Don’t use=0 (from Q8); Less than once a 
month=3.3; 1-3 times a month=6.7; 1-3 times 
a week=10; 4-5 times a week=10 


Score 


Range from 0 to 10 


Q11. Rate how comfortable you are with using the 
Palm Pilots to support your literacy instruction 


Don’t use or Not at all comfortable=0; 2=3.3; 
3=6.7; 4=10; 5 Very comfortable=10 


Score 


Range from 0 to 10 


Q12. Which specific academic foci or instructional 
objectives do you support with the use of handheld 
computers (Palm Pilots) during Targeted Intervention 
instruction? (Check all that apply) 


e Fluency 


e Vocabulary development 


e Developing students’ reading 


comprehension strategies 


Writing skills 


Word parts 


e Word recognition 


Spelling 


Not checked=0; 
Checked=10 
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COMPONENT 5: HIGH QUALITY, HIGH INTEREST MATERIALS - Highly motivating reading materials integrated with 
engaging technology and audio resources. 


Components 


Individual/Summary Items 


Scores 


Total Sub-component 19 Score (Handheld Computers for Targeted Intervention) 


Sub-Component 20: Other 
technology resources 


e Grammar 

e _ Locating information 

e __ Evaluating information 

e _ Synthesizing information 
e Organizing information 


e To develop students’ self-directed learning 


e Teaching students to identify and use the 
organizational features of expository writing 
e _To activate students’ prior knowledge 


Score 


Sum of Above Items/14 (Range from 0 to 
10) 


Q13. Which instructional methods do you support with 
the use of handheld computers (Palm Pilots) during the 
Targeted Intervention instruction? (Check all that 
apply) 

e Monitoring distribution and completion of 
assignments 

e __ Assessing students’ literacy skills 

e Monitoring students’ progress 

e __ Differentiating instruction 

e _ Guided reading 

e Partner reading 
e Individual reading 
e _ Book club disussions 


Not checked=0; 
Checked=10 


Score 


Sum of Above Items/8 (Range from 0 to 10) 


Sum of the Above Items/4 (Range from 0 to 
10) 


LIST Surveys — From 


ELA Teachers 


Q15a. For each of the materials listed below, indicate 
how frequently you currently use the materials to teach 
literacy: 


e _Listening Centers 


N/A (Do Not Have)=0; Not Currently 
Using=0; Less than once a month=2.5; 1 to 3 
times a month=5; | to 3 times a week=7.5; 4 to 
5 times a week=10 


Q15b Rate you comfort level: 


e Listening Centers 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 
4=7.5; 5 Very comfortable=10 


Score 


Sum of Above Items/2 (Range from 0 to 10) 


Q16. For each of the materials listed, please indicate 
which literacy instructional goals are supported by 
your use of that material in your classroom: Listening 
Centers 


e Vocabulary development 
e Fluency 


e __ Reading Comprehension 
e To develop students’ self-directed learning 


e _To-supplement students’ textbook reading 


e _To activate students’ prior knowledge 


Not Using = 0; Checked=10 


Score 


Sum of Above Items/6 (Range from 0 to 10) 


Principal Interviews 
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COMPONENT 5: HIGH QUALITY, HIGH INTEREST MATERIALS - Highly motivating reading materials integrated with 
engaging technology and audio resources. 


Components 


Individual/Summary Items 


Scores 


Q12. For each of the following technology resources 
that your teachers are using to teach literacy, please 
indicate to what extent that technology resource is 
integrated into the literacy curriculum. 


e Listening Centers 


N/A or Not at all integrated=0; Somewhat 
integrated=5; Thoroughly integrated=10 


Score 


Range from 0 to 10 


Total Sub-sub-component 20a Score (Listening Centers) 


Sum of the Above Items/3 (Range from 0 to 
10) 


LIST Surveys — From 


ELA Teachers 


Q15a. For each of the materials listed below, indicate 
how frequently you currently use the materials to teach 
literacy: 

e Media Centers 


N/A (Do Not Have)=0; Not Currently 
Using=0; Less than once a month=2.5; 1 to 3 
times a month=5; | to 3 times a week=7.5; 4 to 
5 times a week=10 


Q15b Rate you comfort level: 
e Media Centers 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 
4=7.5; 5 Very comfortable=10 


Score 


Sum of Above Items/2 (Range from 0 to 10) 


Q16. For each of the materials listed, please indicate 
which literacy instructional goals are supported by 
your use of that material in your classroom: Media 
Centers 


e Vocabulary development 


e _ Fluency 
e Reading Comprehension 


e Writing skills 


e Word Parts 
e Word Recognition 
e Spelling 


° Grammar 


e To develop students’ self-directed learning 


e _Tosupplement students’ textbook reading 

e Teaching students to identify and use the 
organizational features of expository writing 

e _To activate students’ prior knowledge 


Not Using = 0; Checked=10 


Score 


Sum of Above Items/12 (Range from 0 to 
10) 


Principal Interviews 


Q12. For each of the following technology resources 
that your teachers are using to teach literacy, please 
indicate to what extent that technology resource is 
integrated into the literacy curriculum. 

° Media Centers 


N/A or Not at all integrated=0; Somewhat 
integrated=5; Thoroughly integrated=10 


Score 


Range from 0 to 10 


LIST Surveys — From LITs 


Q7a. For each of the materials listed below, indicate 
how frequently you currently use the materials to teach 
literacy: 

e Media Centers 


N/A (Do Not Have)=0; Not Currently 
Using=0; Less than once a month=2.5; 1 to 3 
times a month=5; 1 to 3 times a week=7.5; 4 to 
5 times a week=10 


E-18 


COMPONENT 5: HIGH QUALITY, HIGH INTEREST MATERIALS - Highly motivating reading materials integrated with 
engaging technology and audio resources. 


Components 


Individual/Summary Items 


Scores 


Q7b Rate you comfort level: 


° Media Centers 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 
4=7.5; 5 Very comfortable=10 


Score 


Sum of Above Items/2 (Range from 0 to 10) 


Total Sub-sub-component 20b Score (Media 
Centers) 


Sum of the Above Items/4 (Range from 0 to 
10) 


Total Sub-Component 20 Score (Listening Centers and Media Centers) 


Sum of the Above Sub-sub-Component 
Scores/2 (Range from 0 to 10) 


Total Component 5 Score (Materials) 


(Sum of above Sub-component Scores)/5 = 
Range from 0 to 10 
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COMPONENT 6: Integrated, progressive, high quality professional development 


Components 


Individual/Summary Items 


Scores 


Sub-component 21: 
Whole School 
Professional 
Development 


Professional Development Attendance Records 


The percents below refer to the percent of meetings attended by principals. 


Principals’ Monthly Professional Development 


[<30%]=0; [30-59%]=1; [60-79% ]=2; [80-100%]=3 


The percents below refer to the session attendance rates 


averaged across teachers 


Teachers’ Summer Institute (Yearly) 


[<25%]=0; [25-50%]=1; [51-79%]=2; [80-100%]=3 


Teachers’ Quarterly Follow-Up Institutes 


[<25%]=0; [25-50%]=1; [51-79%]=2; [80-100%]=3 


Total Sub-Component 21a Score- Professional 
Development Attendance 


(Sum of the Above Items/9)*10 


LIST Survey - from ELA Teachers only 


Q42b. For each of the following Striving Readers 
professional development sessions conducted during 
the 2008-2009 school year, please indicate: how useful 
the session(s) was (were) in helping you support 
student learning in language arts. 


Not Participated=missing; Not useful=0; Somewhat 
useful=3.3; Moderately Useful=6.7; Extremely 


2008 Summer institute 
School-year follow-up institutes 


Useful=10 


Technology training (use of handhelds) 


School-based professional development 


Q43b. For each of the following topics, rate the impact 
that professional development you received has had on 
your comfort with each teaching practice. 


Building academic vocabulary 


Classroom libraries 


Creating literacy-rich classroom 
environments 


Differentiating instruction 
Direct vocabulary instruction 


Incorporating text sets in your instruction 


Not Participated=missing; No Impact=0; Some 


Increasing student motivation 


Impact=3.3; Moderate Impact=6.7; Large 


Supporting students’ self-directed learning 


Impact=10 


Using before, during, and after reading 


strategies 


Cc 


sing student assessments to guide 


instruction 
; Using handheld computers (Palm Pilots) 


Ge 


sing literacy-based software 
sing the PRC2 model 


oa) 


Using the whole-part-whole classroom 
instruction model 


Score 


Sum of the Above Items (Range from 0 to 10)/N 
of items with available data 


Principal Interviews 
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COMPONENT 6: Integrated, progressive, high quality professional development 


Components 


Individual/Summary Items 


Scores 


Q16b. For each of the following Striving Readers 
professional development sessions conducted during 
the 2008-2009 school year, please indicate: how useful 
the session(s) was (were) in helping you support 
student learning in language arts. 


Monthly Principal Meetings (Leaders 


Not Participated=missing; Not useful=0; Somewhat 


Seminars) 
: 2008 Summer institute 


School-year follow-up institutes 


Saturday seminars 
On-site training during literacy team 
meetings 


School-based Striving Readers professional 
development 


useful=3.3; Moderately Useful=6.7; Extremely 
Useful=10 


Total Score For Principals 


Sum of the Above Items (Range from 0 to 10)/6 


Total Sub-Component 21b Score - Professional 
Development Survey Responses 


(Sum of the Above Items)/2 = Range from 0 to 10 


Total Sub-Componen 


t 21 Score 


Sum of Above Scores/2 (Range from 0 to 10) 


Sub-Component 22: 


LIST Survey - from ELA Teachers only 


Targeted and 


Professional Development Attendance Records 


Intensive 


The percents below refer to the percent of meetings attended by LIT or principals, respectively. 


Intervention 


LIT Weekly Meetings with Coordinators 


[<60%]=0; [60-74% ]=2.5; [75-89%]=5; [90- 
100%]=10 


Total Score 


Range from 0 to 10 


LIST Survey - from ELA Teachers onl 


Training in LIT/ teacher collaboration 


Not Participated=missing; Not useful=0; Somewhat 
useful=3.3; Moderately Useful=6.7; Extremely 
Useful=10 


Total Score 


Range from 0 to 10 


Total Sub-Component 22 Score 


Sum of Above Scores/2 (Range from 0 to 10) 


Total Component 6 Score - Professional Development 


(Sum of above sub-components)/2 (Range from 0 
to 10) 
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Appendix F: Year 3 Fidelity Scale Results by 
School 


Table F-1 


Results of Year 3 Implementation Fidelity Scales by School 
Major Program Components 


Mean Score 
Component 

Component | Component | Component 4 Component 

Cohort coene racy 1 2 3 Data-Driven onan: 6 
Stare. Blended Targeted Intensive Instruction Materials Professional 
Intervention | Intervention | Intervention & Development 

Assessment 

4 7.0 6.7 8.7 8.2 5.8 5.8 6.8 

5 7.2 6.5 7.9 7.6 8.1 5.6 5.0 

6 7.2 7.2 6.6 7A 8.9 6.1 5.3 

8 7.2 6.3 8.7 8.6 7A 5.1 5.4 

11 7A 7A 73 8.0 7.8 4.8 4.4 

13 7.9 7.2 9.2 8.9 7.8 6.5 6.4 

16 8.0 7.7 8.0 8.1 8.8 7.7 5.3 

17 8.5 8.3 9.5 9.0 7.6 7.9 6.7 

Cohort 19 6.8 6.8 6.8 7.9 6.1 6.1 5.6 

1 20 73 6.4 8.0 7.9 8.3 6.1 5.3 

22 7.7 73 8.4 7.9 8.6 6.1 4.1 

24 6.9 73 8.5 8.6 5.3 5.0 5.8 

27 7A 7.5 8.3 73 T2 5.4 5.6 

29 75 6.7 8.1 9.1 8.7 5.0 4.6 

30 7.8 7.9 9.1 8.7 8.5 4.8 5.8 

31 7.9 7.9 8.0 8.3 8.9 6.2 6.7 

Cohort 1 7.4 7.2 8.2 8.2 7.7 5.9 5.5 

1 7.2 71 8.1 8.1 6.9 6.0 7.7 

2 8.1 7.9 8.8 8.5 7.8 7A 73 

3 7.8 8.9 74 7.6 8.2 6.9 5.3 

7 6.9 7.0 7.9 8.5 5.3 5.8 6.5 

9 7.6 6.9 9.0 8.1 8.4 5.6 7.2 

10 7.7 8.4 9.0 8.3 6.9 6.0 6.9 

12 7.8 8.1 7.6 8.4 73 75 5.3 

Cohort 14 7.2 6.2 9.3 8.3 6.0 6.4 7A 

2, 15 7.8 73 8.7 8.7 8.4 6.1 5.3 

18 7.0 6.7 8.2 8.1 7.8 4.1 6.7 

21 6.9 7.2 75 7.9 6.6 5.4 6.5 

23 6.0 5.8 555 8.3 6.0 4.6 Tal 

25 6.5 72 7A 6.8 4.4 71 7.8 

26 6.9 6.0 9.4 9.0 5.7 4.5 5.1 

28 7.0 Te 74 7.6 6.3 6.5 6.2 

Cohort 2 7.2 7.2 8.1 8.2 6.8 6.0 6.5 

Overall 7.3 Te 8.1 8.2 7.3 5.9 6.0 


' The overall fidelity score is based on components | through 5. 
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Table F-2 


Results of Year 3 implementation fidelity scales by school 
Component 1: Blended intervention 


Mean Score 
Component 1 Rave aeons aut Za ale fee Saas ae eae aa 
era || ee eacs on 
Blended Small Group Gradual Release | Comprehension technology to Marzano’s 
Intervention Instruction Model Focus support Vocabulary 
differentiated 
instruction 

4 6.7 Ted 6.3 6.9 7.2 6.2 
5 6.5 ves) 5.0 8.0 5.6 6.6 
6 7.2 6.9 7.2 75 7.6 6.9 
8 6.3 6.0 6.6 7.8 4.9 6.4 
11 7.4 7.7 6.9 8.0 7.1 74 
13 7.2 75 3.8 8.1 8.7 8.1 
16 7.7 7.8 6.7 7.6 8.1 8.0 
17 8.3 9.3 7.7 8.6 7.9 8.0 
Cohort | 19 6.8 6.5 6.6 75 6.7 6.8 
20 6.4 537 6.3 7.9 5.5 6.5 
22 73 7.3 5.9 8.0 ES 7.8 
24 7.3 7.5 7.1 7.8 8.0 6.0 
27 7a) 8.3 7.8 7.9 75 6.0 
29 6.7 6.8 7.9 6.7 6.2 6.2 
30 7.9 8.0 8.3 8.0 7.6 7.8 
31 7.9 8.6 7.9 8.4 7.7 7.0 
Cohort 1 72 7.4 6.7 7.8 71 7.0 
1 7.1 8.3 6.3 8.2 6.0 6.7 
2 7.9 7.1 8.4 8.9 7.7 7.3 
3 8.9 10.0 8.8 8.5 10.0 ded 
7 7.0 6.8 6.6 7.8 73 6.8 
9 6.9 6.8 6.9 8.2 6.0 6.4 
10 8.4 8.3 8.8 9.0 7.9 7.9 
12 8.1 9.3 Ted: 8.3 TS 8.0 
Cohiort 2 14 6.2 6.6 52, 6.9 5.3 6.8 
15 7.3 8.3 6.8 8.1 6.0 74 
18 6.7 8.0 5.7 7.5 6.0 6.4 
21 7.2 7.7 7.0 7.8 6.6 6.9 
23 5.8 6.0 5.8 6.3 4.6 6.4 
25 7.2 6.7 75 tS 6.4 7.9 
26 6.0 6.5 4.4 7.1 5.6 6.5 
28 2. 6.7 6.7 8.6 5.6 8.4 
Cohort 2 72 75 6.8 7.9 6.6 71 
Overall TA 75 6.8 7.9 6.8 71 
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Component 2: Targeted intervention 


Table F-3 
Results of Year 3 implementation fidelity scales by school 


Mean Score 
Sub- Sub- 
Component 2 Component Component 
Cohort School 6 7] 
Number Direct instruction 
Targeted Teacher/LIT | in comprehension, 
Intervention Collaboration vocabulary and 
fluency 

4 8.7 9.5 8.0 
5 7.9 7.9 8.0 
6 6.6 72 6.0 
8 8.7 9.0 8.5 
11 7.3 5.9 8.7 
13 9.2 9.4 9.0 
16 8.0 74 8.6 
17 9.5 9.3 9.8 
cy 19 6.8 12 6.5 
20 8.0 7.2 8.7 
22 8.4 75 9.3 
24 8.5 8.7 8.3 
27 8.3 8.9 eh 
29 8.1 73 9.0 
30 9.1 9.0 9.3 
31 8.0 74 8.5 
Cohort 1 8.2 8.0 8.4 
1 8.1 8.3 7.9 
2 8.8 9.4 8.1 
3 74 8.1 6.7 
7 7.9 8.4 7.3 
9 9.0 9.1 8.8 
10 9.0 9.6 8.5 
12 7.6 6.6 8.7 
Cohort 14 9.3 9.0 9.6 
2 15 8.7 8.3 9.1 
18 8.2 8.1 8.3 
21 75 7.8 h2 
23 5:35 4.4 6.5 
25 7.1 7.0 2 
26 9.4 9.3 9.5 
28 74 8.3 6.5 
Cohort 2 8.1 8.1 8.0 
Overall 8.1 8.1 8.2 
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Table F-4 


Results of Year 3 implementation fidelity scales by school 
Component 3: Intensive Intervention 


Mean Score 
ae School Component 3 eens Componen t BL ak eee paces da 
Das 4 Increased Explicit Explicit Explicit 
Intensive A Small Group aPalees ama bets 
Tnwrention Instructional Setting (15:1) Instruction in Instruction in Instruction in 
Time Comprehension Vocabulary Fluency 

4 8.2 5.8 10.0 10.0 8.2 7.0 
5 7.6 6.4 10.0 10.0 7.0 4.8 
6 74 7.3 6.8 10.0 5.3 75 
8 8.6 6.1 10.0 10.0 ES 9.2 
11 8.0 5.7 10.0 10.0 7.0 74 
13 8.9 5.8 10.0 10.0 9.5 9.0 
16 8.1 7.2 7.9 10.0 74 8.1 
17 9.0 5.9 10.0 10.0 9.6 9.6 
Cohort | 19 7.9 5.4 10.0 10.0 7.0 7.1 
20 19 4.7 10.0 10.0 8.0 6.8 
22 7.9 7.6 8.8 10.0 6.8 6.5 
24 8.6 5.6 10.0 10.0 8.2 9.2 
27 73 aie) 10.0 10.0 7.0 4.0 
29 9.1 8.3 10.0 10.0 7.9 9.2 
30 8.7 4.3 10.0 10.0 10.0 9.2 
31 8.3 5.4 10.0 10.0 72 8.6 
Cohort 1 8.2 6.1 9.6 10.0 Toll Weal 
1 8.1 5.6 10.0 10.0 7.8 7.1 
y) 8.5 4.9 10.0 10.0 7.8 10.0 
3 7.6 4.8 10.0 10.0 7.0 6.4 
7 8.5 7.6 10.0 10.0 8.2 6.5 
9 8.1 4.5 10.0 10.0 6.7 9.2 
10 8.3 6.3 10.0 10.0 74 8.0 
12 8.4 4.8 10.0 10.0 8.3 8.6 
Cohort 2 14 8.3 Sel 10.0 10.0 8.8 7.9 
15 8.7 6.6 10.0 10.0 8.3 8.6 
18 8.1 5.3 10.0 10.0 7.0 8.1 
21 7.9 4.5 10.0 10.0 75 7.7 
23 8.3 6.7 10.0 10.0 6.9 7.8 
25 6.8 4.9 10.0 10.0 5.9 3.5 
26 9.0 71 10.0 10.0 8.8 9.2 
28 7.6 2.8 10.0 10.0 6.3 9.2 
Cohort 2 8.2 5.4 10.0 10.0 75 7.8 
Overall 8.2 5.8 9.8 10.0 7.6 7.8 
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Component 4: Purposeful Assessment & Data Driven Instruction 


Table F-5 
Results of Year 3 implementation fidelity scales by school 


Mean Score 
Sub- Sub- 
Component 4 Component Component 
Cohort School 13 14 
Number Purposeful 
Assessment & ele Intensive 
: School/Blended 2 
Data Driven aereenion Intervention 
Instruction 
4 5.8 6.7 5.0 
5 8.1 6.2 10.0 
6 8.9 7.9 10.0 
8 7.1 6.6 7.5 
11 7.8 73 8.4 
13 7.8 6.5 9.2 
16 8.8 75 10.0 
17 7.6 8.5 6.7 
cy 19 6.1 64 5.9 
20 8.3 6.7 10.0 
22 8.6 8.0 9.2 
24 5.3 4.8 5.9 
27 7.2 7.8 6.7 
29 8.7 7.4 10.0 
30 8.5 8.6 8.4 
31 8.9 7.9 10.0 
Cohort 1 Tell 7.2 8.3 
1 6.9 7.1 6.7 
2 7.8 8.1 (i) 
3 8.2 6.5 10.0 
7 5.3 74 3.3 
9 8.4 75 9.2 
10 6.9 7.1 6.7 
12 7.3 6.2 8.4 
Cohort 14 6.0 5.4 6.7 
2 15 8.4 77 9.2 
18 7.8 8.1 7.5 
1 6.6 6.5 6.7 
23 6.0 7.8 4.2 
5 4.4 5.5 3.3 
26 5.7 4.8 6.7 
28 6.3 7.6 5.0 
Cohort 2 6.8 6.9 6.7 
Overall 73 7.0 75 
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Results of Year 3 implementation fidelity scales by school 


Table F-6 


Component 5: Materials 


Mean Score 
Sub- Sub- Sub- Sub- Sub- Sub- 
School Component 5 | Component Component Component Component Component Component 
Cohort | Number 15 16 17 18 19 20 
A 1 School Classroom OU ole Handheld ae 
Materials Text sets libra libra technology Ronipiters technology 
y y resources P resources 
4 5.8 10.0 8.1 6.6 ape) 2.6 5.9 
5 5.6 10.0 75 8.5 3.6 3.0 5.6 
6 6.1 10.0 7.1 7.9 3.6 6.1 Duh 
8 5.1 ; 5.2 7.6 3.8 3.0 6.2 
11 4.8 10.0 5.7 7.1 4.8 0 6.4 
13 6.5 j 3.6 8.5 7.6 75 me) 
16 Vel 10.0 7.2 7.9 Fo 7.9 hed: 
17 7.9 10.0 8.7 9.1 8.1 5.3 8.3 
Cohort 1 19 6.1 6.7 5.4 8.3 4.0 5.9 6.9 
20 6.1 6.7 6.2 7.6 3.9 7.1 5.9 
22 6.1 10.0 3.5 8.1 5.7 5.5 7.9 
24 5.0 ‘ 2.0 5.5 2.7 7.7 PA 
27 5.4 6.7 6.0 8.6 6.7 .0 5.6 
29 5.0 10.0 7.6 6.6 5.8 0 4.7 
30 4.8 10.0 1.4 8.8 6.5 0 74 
31 6.2 6.7 8.6 8.5 6.5 .0 74 
Cohort 1 5.9 9.0 5.9 78 5.4 3.9 6.5 
1 6.0 3.3 3.6 8.1 4.1 7.6 6.7 
2 74 10.0 6.1 8.4 8.2 6.6 75 
3 6.9 F 7.6 8.9 8.1 3.5 6.3 
7 5.8 6.7 6.9 6.6 5.0 4.8 6.0 
9 5.6 10.0 5.3 7.6 4.9 4.5 5.8 
10 6.0 10.0 6.9 7.8 8.0 0 74 
12 TD 10.0 8.4 8.8 7.9 5.3 7.3 
Cehon> 14 6.4 : 6.8 7.6 6.3 6.6 4.5 
15 6.1 3.3 3.0 9.0 5.6 6.1 6.7 
18 4.1 : 2.4 6.6 4.3 0 7.2 
21 5.4 10.0 4.2 PS 6.3 4.3 5.0 
23 4.6 6.7 5.7 8.4 4.9 0 3.9 
25 7.1 : 7.7 8.6 7.8 74 4.2 
26 4.5 6.7 1.0 6.4 5.3 5.9 3.9 
28 6.5 10.0 6.4 7.2 4.5 6.3 8.1 
Cohort 2 6.0 79 5.5 78 6.1 4.6 6.0 
Overall 5.9 8.5 5.7 78 5.7 4.2 6.3 


' Sub-Component 15 (Text Sets) is not included in the calculation of component 5 (Materials) or the overall fidelity 


scale due to missing data for 7 of the 31 schools. 
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Table F-7 
Results of Year 3 implementation fidelity scales by school 
Component 6: Professional Development 


Mean Score 
: Component 6 Sub-Component Sub-Component 

Cohort shoal 21 22 

Number : Whole : 

Professional Targeted/Intensive 
Development School Blended Intervention’ 
Intervention 

4 6.8 4.8 6.8 
5 5.0 5.0 5.0 
6 5.3 4.2 5.3 
8 5.4 4.5 5.4 
11 4.4 4.0 4.4 
13 6.4 5.3 6.4 
16 5.3 4.8 5.3 
17 6.7 5.8 6.7 
en 19 5.6 5.0 5.6 
20 5.3 4.5 5.3 
22, 4.1 49 4.1 
24 5.8 4.1 5.8 
27 5.6 4.9 5.6 
29 4.6 3.0 4.6 
30 5.8 Su 5.8 
31 6.7 7 6.7 
Cohort 1 5.5 4.8 5.5 
1 Ted 5.5 7.7 
2 7.3 5.4 7.3 
3 5.3 5.7 5.3 
si 6.5 5.5 6.5 
9 7.2 5.0 7.2 
10 6.9 3.7 6.9 
12 5.3 4.0 5.3 
Cohort 14 7.1 4.8 71 
2 15 5.3 5.0 53 
18 6.7 5.1 6.7 
21 6.5 4.7 6.5 
23 7.1 4.9 7.1 
25 7.8 5.5 7.8 
26 5.1 4.4 5.1 
28 6.2 4.9 6.2 
Cohort 2 6.5 4.9 6.5 
Overall 6.0 4.9 6.0 


> Targeted/Intensive intervention scores are the same as overall PD scores because all components of PD included at 
least some focus on these intervention models. 
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Appendix G: Year 3 Professional Development Schedule 


Summary of Professional Development Sessions 


Target Population Session Duration 
Principals Monthly sessions 3hrs./session X 8 sessions + 
1 7-hr. session + 1 4-day 
conference 
Cohort I & II LITs weekly meetings with 6 hrs./session X 20 sessions 
coordinators 
Cohort II LITs weekly meetings with 6 hrs./session X 4 sessions 
coordinators 
Teachers GoKnow 6 hrs./session X 2sessions 
per participant 
Teachers Teachers’ summer institute 3 days 
Teachers & Saturday seminars 3 hrs./session X 2 sessions 
Administrators 
Teachers, LITs & Quarterly follow-up institutes 3 hrs./session X 4 sessions 
Administrators 
District Coordinators & | Technology Training 6 hrs./session X | sessions 
LITs 
Teachers Technology Training 6 hrs./session X 2 sessions 
Teachers Library Course 3 hrs./session X 6 sessions 
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Professional Development Activities 
Literacy Intervention Teachers (LITs) 
2008-2009 School Year 


Date Duration PD Design Topic Intended # of eligible # 
recipients participants attending 
6/18/2008 6 Comprehension Strategies across the Content Area Cohorts I & I 31 22 
6/18/2008 6 Intervention Data Analysis/Progress Monitoring Cohorts I & II 31 22 
6/20/2008 6 Vocabulary Morphology/word study/vocabulary Cohorts I & II 31 
6/23/2008 3 Intervention Progress Monitoring Cohorts I & I 31 29 
6/23/2008 3 Intervention Intervention Cohorts I & II 31 29 
6/24/2008 6 Intervention Data Analysis/Progress Monitoring Cohorts I & II 31 28 
9/5/2008 6 Intervention Data Analysis/Differentiation Cohorts I & I 31 27 
9/19/2008 3 Comprehension Comprehension Techniques Cohorts I & II 31 28 
10/3/2008 6 Intervention Data/progress monitoring/trends Cohorts I & I 31 25 
10/17/2008 3 Comprehension PRC2 in the Content Area Cohorts I & I 31 26 
10/17/2008 3 Vocabulary PRC2 in Content Area Vocabulary Cohorts I & II 31 26 
10/31/2008 2 Comprehension Successes & Concerns Cohorts I & I 31 26 
10/31/2008 2 Intervention Successes & Concerns Cohorts I & II 31 26 
10/31/2008 2 Vocabulary Successes & Concerns Cohorts I & II 31 26 
11/14/2008 6 Intervention Conferring/Coaching/Progress Monitoring Cohorts I & II 31 27 
12/5/2008 6 Intervention Self-Assessment with intervention Cohorts I & II 31 27 
12/19/2008 2 Comprehension Strategies in wide reading Cohorts I & II 31 26 
12/19/2008 2 Intervention Intervention techniques Cohorts I & I 31 26 
12/19/2008 2 Vocabulary Environmental Word Study Cohorts I & II 31 26 
1/9/2009 2 Comprehension Comprehension in content area Cohorts I & II 31 26 
1/9/2009 2 Intervention Intervention within content area Cohorts I & II 31 26 
1/9/2009 2 Vocabulary Vocabulary in the Content area Cohorts I & II 31 26 
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Date Duration PD Design Topic Intended # of eligible # 
recipients participants attending 

1/23/2009 6 Intervention Intervention Plan Protocol Cohorts I & I 31 28 
2/6/2009 3 Comprehension Comprehension Techniques Cohorts I & I 31 24 
2/6/2009 3 Vocabulary Vocabulary Techniques Cohorts I & II 31 24 
2/18/2009 3 Comprehension Comprehension Strategies Cohorts I & I 31 29 
2/18/2009 3 Vocabulary Vocabulary with Marzano Cohorts I & II 31 29 
3/20/2009 3 Comprehension Comprehension with Technology Cohorts I & I 31 26 
3/20/2009 3 Vocabulary Vocabulary with Technology Cohorts I & II 31 24 
4/3/2009 2 Comprehension Learning to Look at Comprehension Cohorts I & II 31 24 
4/3/2009 2 Vocabulary Learning to Look at Vocabulary Cohorts I & II 31 24 
4/3/2009 6 Pepa: and Needs self- assessment echons i elt 31 24 
5/1/2009 2 Comprehension Evidence of Comprehension Cohorts I & II 31 26 
5/1/2009 2 Intervention Evident of Intervention Cohorts I & I 31 26 
5/1/2009 2 Vocabulary Evidence of Vocabulary Cohorts I & II 31 26 
5/8/2009 35 ner aan and Technology Training olions Teall 50 39 
5/15/2009 6 Intervention Analysis Pre-Post BRI Cohorts I & II 31 25 
5/29/2009 6 Comprehension Action Research Group Cohorts I & II 31 27 
5/29/2009 6 Intervention Action Research Group Cohorts I & II 31 27 
5/29/2009 6 gar ria an Action Planning Cohons-heel e = 
6/25/2009 6 Intervention Intervention Cohorts I & I 31 27 
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Technology Professional Development Activities 
All 6-8 Grade Classroom Teachers, Literacy Intervention Teachers and District Coordinators 
2008-2009 School Year 


Date Duration Topic Intended # of # 
recipients eligible attend 
particip ing 
ants 

9-10- 3 hours GoKnow Software Teachers 31 22 

2008 Applications 

9-23- 3 hours GoKnow Software Teachers 35 29 

2008 Applications 

9-24- 3 hours GoKnow Software Teachers 75 52 

2008 Applications 

9-25- 3 hours GoKnow Software Teachers 75 58 

2008 Applications 

9-26- 3 hours GoKnow Software Teachers 75 53 

2008 Applications 

10-1- 3 hours GoKnow Software Teachers 31 20 

2008 Applications 

10-24- 3 hours GoKnow Software Teachers 75 47 

2008 Applications 

1-14- 3 hours GoKnow Software Teachers 28 28 

2009 Applications 

1-15- 3 hours GoKnow Software Teachers 32 34 

2009 Applications 

1-16- 3 hours GoKnow Software Teachers 34 38 

2009 Applications 

1-17- 3 hours GoKnow Software Teachers 30 33 


2009 Applications 


1-18- 
2009 


5-08- 
2009 


Date 


8-25- 
2008 


10-15- 
2008 


10-15- 
2008 


12-8- 
2008 


12-8- 
2008 


2-20- 
2009 


4-15- 
2009 


4-16- 
2009 


5-21- 


3 hours GoKnow Software Teachers 29 
Applications 


6 hours GoKnow Software Coordinators 50 


Applications 


Professional Development Activities 
All Striving Readers Treatment Schools Principals 
2008-2009 School Year 


Time PD Design Topic # of 

allotted eligible 

to the particip 

session ants 

3 hours Comprehension Wide Reading 31 

1.5 hours Comprehension Instructional 31 

Leader 
1.5 hours Intervention Coaching toward 31 
understanding 
1.5 hours Comprehension Learning to Look 31 
at 

Comprehension 

1.5 hours Vocabulary Learning to Look 31 

at Vocabulary 

3 hours Comprehension Content Area 31 
Comprehension 

3 hours Intervention Implementation 31 

Rubrics 

3 hours Intervention Data Analysis 31 
with technology 

3 hours Intervention Reflection with 31 


35 


39 


# 
attendi 
ng 
15 
22 


22 


22 


22 


24 
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2009 


5-27- 
2009 


Date 


ine 
2008 
(152 
2008 
O15: 
2009 
4-2- 
2009 
5-6- 
2009 
6-3- 
2009 


3 hours 


Intervention 


Data Analysis 


Chicago Striving 


Readers Logic 


Library Professional Development Activities 


Time 
allotted 
to the 
session 
3 hours 
3 hours 
3 hours 
3 hours 


3 hours 


3 hours 


Librarians 


2008-2009 School Year 


Topic 


School Library 
Leadership 


School Library 
Leadership 


School Library 
Leadership 


School Library 
Leadership 


School Library 
Leadership 


School Library 
Leadership 


# of 
eligible 
particip 

ants 

15 
15 
15 
15 


15 


15 


31 21 


# 
attendi 
ng 


12 


11 


11 


G-6 


Saturday Seminars - Professional Development 
All 6-8 Grade Classroom and Resource Teachers, Literacy Intervention Teachers, Principals and District Coordinators 
2008-2009 School Year 


Date 


2-23- 


2008 


3-29- 
2008 


Time 
allotted 
to the 
Session 


3 hours 


3 hours 


# of 
eligible 
participan 
ts 


100 


100 


# 
atten 
ding 


111 


108 


Summer Institute - Professional Development 
All 6-8 Grade Classroom and Resource Teachers, Literacy Intervention Teachers, Principals and District Coordinators 
2008-2009 School Year 


Date 


6-18- 
2008 


6-19- 
2008 
6-20- 
2008 


Time 
allotted 
to the 
Session 


6 hours 


6 hours 


6 hours 


# of 
eligible 
participan 
ts 


250 


250 


250 


# 
atten 
ding 


211 


215 


176 
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Follow up Institutes - Professional Development 


All 6-8 Grade Classroom and Resource Teachers, Literacy Intervention Teachers, Principals and District Coordinators 
2008-2009 School Year 


Date 


9-20- 
2008 
jiergs 
2008 
11-22- 
2008 
aie 
2009 
4-25- 
2009 
4-25- 
2009 


Place 


Medill 


Smyth 


Smyth 


Smyth 


Smyth 


Smyth 


Time 
allotted to 
the session 


1.5 Hours 


1.5 Hours 


1.5 Hours 


3 Hours 


1.5 Hours 


1.5 Hours 


PD Design 


Comprehension 


Comprehension 


Intervention 


Comprehension 


Comprehension 


Intervention 


Topic 


Small Group 


Differentiation 


Techniques 


Highlight 


Number 
of 
eligible 
particip 
ants 
100 
100 
100 
100 
100 


100 


Actual 
number of 
participants 


82 


66 


66 


125 


86 


86 
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Appendix H: Year 3 Principal Interviews, Selected 


Comparisons of Treatment and Control Schools 


Table H-1 


Quality of the Literacy Team’s Performance in the Following Areas 


N and % of Principals Responding 


level teams. 


Addressing the needs of individual 


Treatment Control 

Performance Area* 
a ri Excel- a Excel- 

N Poor Fair Good lent N Poor Fair Good lent 
Addressing the needs of all | 57 | 99% 11% | 48% | 41% || 22 | 0% 5% 36% 59% 
students 
eee the needs of struggling | 37 | gq 4% 48% | 48% || 22 | o% 9% 41% 50% 
Addressing. We. Meeds: OF Orage seit 8% 28% | 60% || 25 | 12% | 4% 40% 44% 


your school.* 


27 | 4% 7% 52% 37% 25 | 12% 16% 28% 44% 
teachers 
age atte 4 been 27 | 0% 0% 48% | 52% || 25 | 12% | 4% 32% 52% 
Using assessment data and or | 57 | go 11% | 41% | 48% | 25 | 12% | 4% 36% 48% 
student work to drive instruction 
Supporting vertical and_ horizontal 26 | 4% 3% 50% 38% 25 | 120% 12% 40% 26% 
teacher collaboration 
Improving literacy instruction at 27 | 0% 70% 26% 67% 25 | 12% A% 48% 36% 


* Schools where principals reported that there was no literacy team, or where the literacy team has not yet met have been recoded as 


“poor.” 


*An asterisk in this column denotes a statistically significant difference (9 <.05) between Treatment and Control responses based on a 


Mann-Whitney U test. 


Table H-2 


Extent School Uses Assessment Data for the Following Purposes 


N and % of Principals Responding 


Treatment Control 
Purpose* Toa Toa 
Not at foie moder- ‘Tore Not at fa moder- Tee 
N all small Ae large N all small Ae large 
extent erent extent extent Etent extent 
Screening students ability levels for 27 0% A% 19% 78% 26 0% 0% 35% 65% 
placement in intervention programs 
Diagnosing students’ strengths and 
support needs for placement in specific 27 0% 4% 26% 70% 25 0% 4% 36% 60% 
courses or instructional groups 
Identifying trends in fluency and 
comprehension abilities across groups 27 0% 11% 19% 70% 26 0% 4% 50% 46% 
of students 
EMU I Hens Ih VObabUlary 27 0% 11% 22% 67% 26 0% 8% 54% 38% 
knowledge across groups of students 
Monitoring overall student progress for 
the purpose of assessing success of 26 0% 0% 19% 81% 26 0% 0% 23% 77% 
instructional programs and methods 
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N and % of Principals Responding 


Treatment Control 
Purpose* Toa Toa 
Not at ie moder- oe Not at Ws moder- ee 
N small large N small large 
all ate all ate 
extent extent extent extent 
extent extent 
Differentiating instruction 26 4% 8% 35% 54% 26 0% 8% 31% 62% 
Riri ois Poles sional 27 0% 4% 22% 74% 26 0% 0% 35% 65% 
evelopment 


*An asterisk in this column denotes a statistically significant difference (p <.05) between Treatment and Control responses based on a 
Mann-Whitney U test. 


Table H-3 
Extent Non-Literacy Teachers Integrate Literacy into the Following Subjects 


N and % of Principals Responding 
Treatment Control 
Subject Area* To a To a 
Not at ioe ae moder- aks Not at joa moder- a 
N small large N small large 
all ate all ate 
extent extent extent extent 
extent extent 
Mathematics 27 0% 19% 52% 30% 26 4% 12% 58% 27% 
Social Studies 27 0% 4% 33% 63% 26 0% 4% 27% 69% 
Science 27 0% 11% 37% 52% 26 0% 12% 42% 46% 


*An asterisk in this column denotes a statistically significant difference (p <.05) between Treatment and Control responses based on 
a Mann-Whitney U test. 


Table H-4 
Use of the following Technology Resources for Lit. Instruction 


N and % of Principals 


Responding 
Resource* Treatment || Control 
N Yes N Yes 
Media Centers 27 | 96% 26 96% 
Listening Centers 26 | 96% 26 92% 
Handheld Computers / Laptops (Control)* 26 | 100% 25 52% 


*An asterisk in this column denotes a statistically significant difference (p <.05) between Treatment and Control responses based on 


a Chi-Square test. 


Table H-5 


Extent the Following Tech Resources are Integrated into Literacy Curricula 


Resource* 


N and % of Principals Responding 


Treatment Control 
N Not at all | Somewhat | Thoroughly N Not at all | Somewhat | Thoroughly 
integrated 2 integrated integrated integrated ; integrated integrated 


/ Laptops (Control)* 


Media Centers* 27 | 4% 33% 63% 26 | 4% 69% 27% 
Listening Centers 27 4% 56% 41% 26 | 8% 54% 38% 
Handheld Computers | 57 | 40g 59% 37% 22 | 50% 41% 9% 


* Schools where principals reported not having or not using the technology resource were recoded as “not at all integrated.” 
*An asterisk in this column denotes a statistically significant difference (p <.05) between Treatment and Control responses based on 
a Mann-Whitney U test. 


Table H-6 
Overall Integration of Technology into Literacy Curricula 
N and % of Principals Responding 
Treatment Control 
N Not at all | Somewhat | Thoroughly N Not at all | Somewhat Thoroughly 
integrated integrated integrated integrated integrated integrated 
Overall, how well is 
technology” AME UIaIeO 155.1) 006 44% 56% 25 | 0% 68% 32% 
into the literacy 
curriculum? 


Note: No statistically significant difference (9 <.05) between Treatment and Control responses based on a Mann-Whitney U test 


was found. 
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Appendix I: Definitions of 


Year 4 Fidelity Scores 


COMPONENT 1: Reading comprehension instruction for WHOLE SCHOOL, BLENDED INTERVENTION 


Sub-Components 


Individual/Summary Items 


Scores 


Sub-Component 1: 
Individual and 
small group 
instruction 


LIST Survey - fr 


om ELA Teachers only 


Q6. How often do you use the following grouping structures 
in your classes? 


Never=0; Less than once a month=2; 1-3 times a month=4; 1-3 times 


Individual Work 


a week=6; 4-5 times a week=8; Multiple times a day=10 


Small groups or Pairs 


Score 


Sum of the Above Items/2 (Range from 0 to 10) 


Q16. For each of the materials listed below, please indicate 
which grouping strategies are supported by your use of that 
material in your classroom. 


Listening centers 


For each item: 


Media Centers 


=0 if small group AND individual work are NOT checked 


Classroom Library 


=5 if either small group OR individual work are checked 
=10 if BOTH small group AND individual work are checked 


Vocabulary Notebooks 


Reading response notebooks 


Score 


Sum of the Above Items/5 (Range from 0 to 10) 


Total Sub-Component 1 Score (Individual and Small 
Group Instruction) 


Sum of the Above Scores/2 (Range from 0 to 10) 


Sub-Component 2: 
Instruction anchor 
for all classrooms 


LIST Survey - fr 


‘om ELA Teachers only 


Q2. How often do you use the following practices to help 
students increase reading comprehension? 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 


I-1 


and content areas Explicit instruction in the use of any one or more of the 
is focused on following comprehension strategies: summarizing, 
comprehension. questioning, predicting, text structure, visualization, 
inferring and metacognition 


Score Range from 0 to 10 


Q2. How often do you use the following practices to help 
students increase reading comprehension? 


- Establishing the purpose for reading. 

- Monitoring students’ comprehension through 
questioning. 

- Use of before, during, and after (BDA) reading 
strategies for comprehension instruction 


Never=0; Less than once a month=2.5; 1-3 times a month=5; 1-3 
times a week=7.5; 4-5 times a week=10; 


Q2. How often do you use the following practices to help 
students increase reading comprehension? 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 


Making connections to background knowledge. Gig naresk 1014-94 mes 4 we 10: 


Making connections between texts. 


Synthesizing information within text or across texts. 


Score Sum of the Above Items/6 (Range from 0 to 10) 


Q5. How often do you use the following techniques to help 
struggling readers develop better reading strategies and 
skills? 


Everybody Reads To (ERT) 
Exclusion Brainstorming 
List-Group-Label 


Interactive Notation System for Effective Reading 
and Thinking (INSERT) 


Guided Reading and Summarizing Procedure 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 


(GRASP) 
a Never/ Not Familiar=0; Less th th=5; 1-3 ti 
: ever/ Not Familiar=0; Less than once a month=5; 1-3 times a 
Eig Ear eee (EEAN) month=10; 1-3 times a week=10; 4-5 times a week=10 
ABC Graffiti 
KWL 


Never/ Not Familiar=0; Less than once a month=3.3; 1-3 times a 
month=6.7; 1-3 times a week=10; 4-5 times a week=10 


Score Sum of the Above Items/8 (Range from 0 to 10) 


Q15. For each of the materials listed across the top of the 
chart below, please indicate which literacy instructional 
goals are supported by your use of that material in your 
classroom: Reading Comprehension 


Listening centers 


Media centers 


Classroom library 


Not checked = 0; Checked=10 


Score 


Sum of the Above Items/3 (Range from 0 to 10) 


Total Sub-Component 2 Score (Systematic Comprehension) 


Sum of the Above Scores/4 (Range from 0 to 10) 


Sub-Component 3: 
PRC2 instructional 
framework, text 
sets, and 
technology are 
used fluidly and 
alternately to 
support 
differentiated 
instruction and 
increase student 
motivation, 
engagement, and 
understanding. 


LIST Survey - from ELA Teachers only 


Q2. How often do you use the following practices to help 
students increase reading comprehension? 


PRC2 


Q3. How often do you use the following practices to help 
students build their vocabulary knowledge? 


PRC2 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 


Q2. How often do you use the following practices to help 
students increase reading comprehension? 


Using differentiated instruction 


Q7. Considering your own instruction, how often do you apply 
differentiated instruction in your classroom? 


Never=0; Rarely=2.5; Occasionally=5; About half the time=7.5; Most 
of the time=10; Almost every lesson or activity=10 


Q16 Use of Listening Centers for differentiating instruction for 
struggling readers 


Q16 Use of Media Centers for differentiating instruction for 
struggling readers 


Not checked = 0; Checked=10 


Q21 Use of Palm Pilots for differentiating instruction for 
struggling readers 


Total Sub-Component 3 Score (Use of PRC2, Text Sets and 
Technology for Differentiated Instruction) 


Sum of the Above Items/7 (Range from 0 to 10) 


Sub-Component 4: 
Systematic 
approach to 
teaching academic 
content vocabulary 
in all subjects using 


LIST Survey - fr 


om ELA Teachers only 


Q3. How often do you use the following practices to help 
students build their vocabulary knowledge? 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 


Explicit instruction in vocabulary 


Modeling the use of word parts 


Review of vocabulary words 
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Robert Marzano’s 
Building Academic 
Content Vocabulary 


Use of vocabulary notebooks 


Use of before, during, and after (BDA) reading 
strategies for vocabulary instruction 


Words Their Way 


Academic Vocabulary for content terms 


Word study sorts and concepts 


Morphology instruction 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 


Score 


Sum of the Above Items/9 (Range from 0 to 10) 


Q15. For each of the materials listed across the top of the 
chart below, please indicate which literacy instructional 
goals are supported by your use of that material in your 
classroom: Vocabulary Development 


Listening centers 


Media centers 


Classroom library 


Vocabulary notebooks 


Checked =0; Not Checked =10. 


Score 


Sum of the Above Items/4 (Range from 0 to 10) 


Total Sub-Component 4 Score (Vocabulary) 


Sum of the Above Scores/2 (Range from 0 to 10) 


Total Component | - Blended Intervention Score: 


Sum of the Above Sub-Components/5 (Range from 0 to 10) 
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COMPONENT 2: Reading comprehension instruction for TARGETED intervention model for Tier 2 and TIER 3 students 


Sub-Components 


Sub-Component 5: 


Teachers and 
Literacy 
Intervention 
Teachers 
collaboration in 
instructional 
planning and 
progress 
monitoring. 


Individual/Summary Items 


Scores 


LIST Survey — from LIT only (about targeted intervention) 


Q8. How often do you meet with ELA classroom teachers at 
the following grade levels to discuss instruction-related issues 
regarding your work with students in the Targeted 
intervention group. 


e Grade 6 teachers 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10 


e Grade 7 teachers 


e Grade 8 teachers 


Never=0; Less than once a month=5; 1-3 times a month=10; 1-3 times 
a week=10; 4-5 times a week=10 


Score 


Sum of the Above Items/3 (Range from 0 to 10) 


Q9. In which setting(s) do you meet or collaborate with ELA 
classroom teachers? (Check all that apply for each grade) 


e Grade 6 teachers 


e Grade 7 teachers 


e Grade 8 teachers 


For Q9a, 9b, and 9c: 

Sum of the following (for a total score of 10): Scheduled one-on-one 
meetings = 3.3 or 0 (if not checked); Grade-level team meetings = 3.3 
(if checked) or 0 (if not checked); Literacy leadership team meetings 
=3.3 (if checked) or 0 (if not checked) 


Score 


Sum of the Above Items/3 (Range from 0 to 10) 


Q10. How often do you meet with SIXTH-GRADE classroom 
teachers to discuss implementing each of the following 
instructional methods for students in the in-class Targeted 
Intervention group (Tiers 2-3)? 


e Differentiated instruction 


e Student groupings 


e = Use of Striving Readers texts sets, text set teacher 
guides, technology, classroom library, school library 


Never=0; Less than once a month=5; 1-3 times a month=10; 1-3 times 
a week=10; 4-5 times a week=10 
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e = Use of specific Striving Readers comprehension 
strategies for reading 


e Using specific Striving Readers instructional 
techniques for comprehension instruction 


e Using specific Striving Readers instructional 
techniques for vocabulary instruction 


e Using specific Striving Readers instructional 
techniques for fluency instruction 


e _ Discussing specific students’ reading progress 


e = Coordinating instruction between lessons for the 
whole class and lessons for the Targeted Intervention group 


e Using student assessment data for instructional 
planning 


Score 


Sum of the Above Items/10 (Range from 0 to 10) 


LIST Survey — 


from ELA Teachers 


Q35. How often do you meet or collaborate with the LIT in the 
following settings? 


a) Scheduled one-on-one meetings 


c) Grade-level team meetings 


For 6th grade teachers: Never=0; Less than once a month=3.3; 1-3 
times a month=6.7; 1-3 times a week=10; 4-5 times a week=10 


For 7th or 8th grade teachers: Never=0; Less than once a month=5; 1- 


3 times a month=10; 1-3 times a week=10; 4-5 times a week=10 


Score 


Sum of the Above Items/2 (Range from 0 to 10) 


Q36. To what extent has your collaboration with the LIT 
facilitated your efforts to use the following methods to support 
struggling readers in your class? 


e Differentiating instruction 


e Scaffolding of instruction 


e Student groupings 


e Using the media center 


e __ Using listening centers 


Not at all=0; To a small extent=5; To a moderate extent=10; To a 
large extent=10. 


(Also 0 if Q38a and 38c=0) 
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e Using assessment data to monitor student progress 


e Using student assessment data for instructional 
planning 


Score 


Sum of the Above Items/8 (Range from 0 to 10) 


Q37. To what extent has your collaboration with the LIT 
facilitated your ability to provide effective instruction in the 
following areas for struggling readers? 


e Comprehension 


e Fluency 


e Vocabulary 


e = Writing skills 


e Word parts 


e Word recognition 


e = Spelling 


e _ Reading/literacy in content areas 


Not at all=0; To a small extent=5; To a moderate extent=10; To a 
large extent=10. 


(Also 0 if Q38a and 38c=0) 


Score 


Sum of the Above Items/8 (Range from 0 to 10) 


Sub-Component 6: 


Total Sub-Component 5 Score (Collaboration) 


Sum of the Above Scores/6 (Range from 0 to 10) 


Explicit 
instruction in 


Q1 How often do you use the following grouping structures 
during the push-in intervention with Tier 2and 3 students? 


small group setting 


e Small groups/Pairs 


Never=0; Less than once a month=2.5;1-3 times a month=5;1-3 times 
a week=7.5;4-5 times a week=10; Multiple times a day=10 


for Tier 2-3 
students for 
approximately 20- 
30 minutes per 
day, in 7 core 
comprehension 
strategies: 


Q2 During your work in the regular classroom with 
students in the Targeted intervention group (Tier 2 and 3 
students), how often do you apply differentiated instruction 
(providing different content, resources and/or instructional 
techn iques specifically tailored to meet students' individual 
educational needs and/or learning styles?) 


Never=0, Rarely=2.5, Occasionally=5, About half the time=7.5, Most 


of the time=10, Almost every lesson or activity=10 


summarization, 


Score 


Sum of the Above Items/2 (Range from 0 to 10) 


predicting, 
inferring, 
metacognition, 
visualization, 


Q3. During your work in the regular classroom with 
students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following practices to help 
struggling readers increase reading comprehension? 
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questioning, and 
text structure. 
[Also addresses 
grouping, 
vocabulary 
instruction, fluency 


° Explicit instruction in the use of any one or more of 
the following comprehension strategies: summarizing, 
questioning, predicting, text structure visualization, inferring 
and metacognition 


Never=0; Less than once a month=3.3;1-3 times a month=6.7;1-3 
times a week=10;4-5 times a week=10; Multiple times a day=10 


Score 


Range from 0 to 10 


Q3. During your work in the regular classroom with 
students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following practices to help 
struggling readers increase reading comprehension? 


e __ Establishing the purpose for reading 


e Monitoring students’ comprehension through 
questioning 


e Making connections to background knowledge 


e Making connections between texts 


Never=0; Less than once a month=2.5;1-3 times a month=5;1-3 times 


e _Synthesizing information within text or across texts 


e Using differentiated instruction 


e Use of BDA reading strategies for comprehension 
instruction 


a week=7.5;4-5 times a week=10; Multiple times a day=10 


Q2. During your work in the regular classroom with 
students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following practices to help 
struggling readers increase reading comprehension? 


* Using Pairing for Partner Reading & Content Too 
(PRC2) for comprehension instruction 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 


Q6. During your work in the regular classroom with 
students in the Targeted intervention group, in a typical 
classroom, to what extent do you feel you are able to meet your 
Tier 2 and 3 students’ individual needs through these 
instructional practices? 


Not Using = 0; Not at all = 0; To some extent = 3.3; To a moderate 
extent = 6.7, To a large extent = 10 


oS Use of the gradual release of responsibility model for 

reading comprehension instruction 

Score Sum of the Above Items/9 (Range from 0 to 10) 
Score Sum of Scores 16 and 17/2 (Range from 0 to 10) 
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Q4. During your work in the regular classroom with 
students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following practices to help 
struggling readers build their vocabulary knowledge? 


e Explicit instruction in vocabulary 


e Modeling the use of word parts 


e Use of the PRC2 for vocabulary development 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 


Q4. During your work in the regular classroom with 
students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following practices to help 
struggling readers build their vocabulary knowledge? 


e __ Review of vocabulary words 


e Use of vocabulary notebooks 


e Use of BDA reading strategies for vocabulary 
instruction 


Never =0; Less than once a month=2.5; 1-3 times a month=5; 1-3 
times a week=7.5; 4-5 times a week=10 


Q4. During your work in the regular classroom with 
students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following practices to help 
struggling readers build their vocabulary knowledge? 


e Words Their Way 


e Academic Vocabulary for content terms (e.g., 
Marzano) 


e Word study- word sorts and concepts (e.g., Donald 
Bear) 


e Morphology instruction (e.g., Shane Templeton) 


Never =0; Less than once a month=2.5; 1-3 times a month=5; 1-3 
times a week=7.5; 4-5 times a week=10 


Q6. During your work in the regular classroom with 
students in the Targeted intervention group, in a typical 
classroom, to what extent do you feel you are able to meet your 
Tier 2 and 3 students’ individual needs through these 
instructional practices? 


Not Using = 0; Not at all = 0; To some extent = 3.3; To a moderate 
extent = 6.7, To a large extent = 10 


e Use of the gradual release of responsibility model to 
build vocabulary 
Score Sum of the Above Items/11 (Range from 0 to 10) 
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Q5. During your work in the regular classroom with 
students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following practices to help 
struggling readers develop fluency? 


e __ Teacher interactive read aloud 


e Shared reading 


Never =0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10 


Q6. During your work in the regular classroom with 
students in the Targeted intervention group, in a typical 
classroom, to what extent do you feel you are able to meet your 
Tier 2 and 3 students’ individual needs through these 
instructional practices? 


e Students listen to audio books, play aways 


Not Using = 0; Not at all = 0; To some extent = 3.3; To a moderate 
extent = 6.7, To a large extent = 10 


Q5. During your work in the regular classroom with 
students in the Targeted intervention group (Tier 2 and 3 
students), how often do you use the following practices to help 
struggling readers develop fluency? 


e _ Teacher read aloud 


e Modeling reading for students 


Never =0; Less than once a month=2.5; 1-3 times a month=5; 1-3 
times a week=7.5; 4-5 times a week=10 


Q6. During your work in the regular classroom with 
students in the Targeted intervention group, in a typical 
classroom, to what extent do you feel you are able to meet your 
Tier 2 and 3 students’ individual needs through these 
instructional practices? 


e — Explicit instruction in guided oral reading 


Not Using = 0; Not at all = 0; To some extent = 3.3; To a moderate 
extent = 6.7, To a large extent = 10 


oS Use of the gradual release of responsibility model to 
develop fluency 
Score Sum of the Above Items/7 (Range from 0 to 10) 


Q7 How often do you use the following techniques to help 
struggling readers develop better reading strategies and 
skills? 


e ReQuest 


e =6©ABC Graffiti 


e __ Predict-Locate-Add-Note (PLAN) 


Never =0; Less than once a month=5; 1-3 times a month=10; 1-3 
times a week=10; 4-5 times a week=10 


Q5 How often do you use the following techniques to help 
struggling readers develop better reading strategies and 
skills? 


Never =0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10 
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e Everybody Reads To (ERT) 


e _ Exclusion Brainstorming 


e _List-Group-Label 


e Interactive Notation System for Effective Reading 
and Thinking (INSERT) 


e Guided Reading and Summarizing Procedure 
(GRASP) 


e KWL 


Score 


Sum of the Above Items/9 (Range from 0 to 10) 


Total Sub-Component 6 Score (Comprehension) 


Total Component 2 - Targeted Intervention Score: 


Sum of the Above Scores/5 (Range from 0 to 10) 


(Sum of above Sub-component Scores)/2 = Range from 0 to 10 


COMPONENT 3: Reading comprehension instruction for intensive intervention model for Tier 3 students 


Sub-Components 


Sub-Component 7: 
Increased time— 
an additional 240 
minutes of direct 
and supported 
instruction beyond 
the intervention 
that occurs during 
the regular school 
day. 
Sub-Component 8: 
Small groups 
setting: 15 to 1 
teacher student 
ratio. 


Sub-Component 9: 
Explicit and 
systematic 
instruction in 
seven core 
comprehension 
strategies: 
summarization, 
predicting, 
inferring, 
metacognition, 
visualization, 
questioning, and 
text structure 
(strategies 
introduced one at 
a time) during the 
additional 240 
minutes of 
supported 
instruction. 
Sub-Component 


Individual/Summary Items 


Scores 


AMP Schedule and Attendance Records 


Total # minutes AMP Classes should meet per week (240) * 
Number of Weeks (26) = 6240 minutes 


Score = (Total Number of Minutes the Program Operated at Each 
School)/624, max=10 


Average number of minutes attended by student by school. 


Score = (Total Number of Minutes Attending Averaged Across 
Students)/624, max=10 


Total Sub-Component 7 Score (Increased Time) 


Sum of above Items/2 (Range from 0 to 10) 


AMP Enrollment Records 


Number of Students and Teachers Per Class 


(# Teachers)/(# Students) X 150, max=10 


Total Sub-Component 8 Score (Small Group Setting) 


Range from 0 to 10 


LIST Survey - from LIT and AMP teachers only (section about intensive intervention) 


Q9. How often do you use the following practices or materials 
with Tier 3 students in the AMP after-school program to help 
them increase reading comprehension? 


° Explicit instruction in the use of any one or more of 
the following comprehension strategies: summarizing, 
questioning, predicting, text structure visualization, inferring 
and metacognition 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10 


Q12. During your work with students in the AMP after- 
school program, in a typical classroom, to what extent do you 
feel you are able to meet your Tier 3 students’ individual needs 
through these instructional practices? 


oS Use of the gradual release of responsibility model for 
reading comprehension instruction 


Not Using = 0; Not at all = 0; To some extent = 3.3; To a moderate 
extent = 6.7, To a large extent = 10 


Total Sub-Component 9 Score (Systematic 
Comprehension) 


Sum of above Items/2 (Range from 0 to 10) 


LIST Survey - from LIT and AMP teachers only (section about intensive intervention) 
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10: Teaching of 
high volume and 
depth of academic 
vocabulary. 


Sub-Component 
11: Guided fluency 
practice. 


Q10. How often do you use the following practices or 
materials with Tier 3 students in the AMP afterschool program 
to help them build their vocabulary knowledge? 


e Explicit instruction in vocabulary 


e Modeling the use of word parts 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10 


e _ Review of vocabulary words 


e = Use of before, during, and after (BDA) reading 
strategies for vocabulary instruction 


Never=0; Less than once a month=2.5; 1-3 times a month=5; 1-3 
times a week=7.5; 4-5 times a week=10; 


e Academic vocabulary for content terms 


e Word study-word sorts and concepts 


e — Morphology instruction 


Never=0; Less than once a month=2.5; 1-3 times a month=5; 1-3 
times a week=7.5; 4-5 times a week=10; 


Q12. During your work with students in the AMP after- 
school program, in a typical classroom, to what extent do you 
feel you are able to meet your Tier 3 students’ individual needs 
through these instructional practices? 


oS Use of the gradual release of responsibility model to 
build vocabulary 


Not Using = 0; Not at all = 0; To some extent = 3.3; To a moderate 
extent = 6.7, To a large extent = 10 


Total Sub-Component 10 Score (Vocabulary) 


Sum of the Above Items/8 (Range from 0 to 10) 


LIST Survey - from LIT and AMP teachers only (section about intensive intervention) 


Q11. How often do you use the following practices or 
materials with Tier 3 students in the AMP afterschool program 
to help them develop fluency? 


e __ Teacher interactive read aloud 


e Shared reading 


e Students listen to audio books, play aways 


Never =0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10 


Q11. How often do you use the following practices or 
materials with Tier 3 students in the AMP afterschool program 
to help them develop fluency? 


e Teacher read aloud 


e Modeling reading for students 


Never =0; Less than once a month=2.5; 1-3 times a month=5; 1-3 
times a week=7.5; 4-5 times a week=10 
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Q12. During your work with students in the AMP after- 
school program, in a typical classroom, to what extent do you 
feel you are able to meet your Tier 3 students’ individual needs 
through these instructional practices? 


e Use of the gradual release of responsibility model to 
develop fluency 


oS Explicit instruction in guided oral reading to develop 
fluency 


Not Using = 0; Not at all = 0; To some extent = 3.3; To a moderate 
extent = 6.7, To a large extent = 10 


Total Sub-Component 11 Score (Guided fluency practice) 


Total Component 3 - Intensive Intervention Score: 


Sum of the Above Items/7 (Range from 0 to 10) 


(Sum of above Sub-component Scores)/5 = Range from 0 to 10 
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COMPONENT 4: Frequent, purposeful assessment and adjustment of instruction with screening, diagnostic, and progress-monitoring 
tools and data-driven instruction structured through a team-based system of leadership and support 


Sub-Component 
12: Purposeful 
assessments with 
screening, 
diagnostic, and 
progress- 
monitoring tools 
and data-driven 
instruction 
structured through 
a team-based 
system of 
leadership and 
support FOR 
WHOLE 
SCHOOL 
BLENDED 
INTERVENTION 


Principal Interviews 


Q5. To what extent is student assessment data being used 
for this purpose? 


e Screening students’ ability levels for placement in 
intervention programs 


e Diagnosing students’ strengths and support needs for 
placement in specific courses or instructional groups 


e Identifying trends in fluency and comprehension 
abilities across groups of students 


e Identifying trends in vocabulary knowledge across 
groups of students 


e Monitoring overall student progress for the purpose 
of assessing success of instructional programs and methods 


e __ Differentiating instruction 


e Planning on-site professional development 


Not at all=0; To s Small Extent=3.3; To a Moderate Extent=6,7; To a 
Large Extent=10 


Score 


Sum of above Items/7 (Range from 0 to 10) 


Q3c. Overall, rate the quality of the literacy team’s 
performance in the following areas 


e Using assessment data and or student work to plan 
instruction 


[If school does not have a literacy team=0] Poor=0; Fair=3.3; 
Good=6.7; Excellent=10; Not Sure=0 


Score 


Range from 0 to 10 


Q4c. Overall, rate the quality of the grade level team’s 
performance in the following areas 


e _ Using assessment data to plan instruction 


e Using assessment data to establish vertical and 
horizontal literacy goals by grade level 


[If school does not have a grade-level team=missing] Poor=0; 
Fair=3.3; Good=6.7; Excellent=10; Not Sure=0 


Score 


Sum of above Items/2 (Range from 0 to 10) 


LIST Surveys — from ELA Teachers 


Q8 Indicate how you use the data from the following 
assessments: 


[From Not used=0 to Used in all intended ways=10. No extra points 
for additional applications. ] 


Reading Benchmark Assessment 


Not using=0; Screening = +2.5; Benchmarking=+2.5; Progress 
monitoring= +2.5 Assess outcomes=+2.5 


ISAT 


Not Using=0; Screening=+5; Outcome=+5 


Informal assessments 


Not using=0; Diagnostic = +2.5; Benchmarking=+2.5; Progress 
monitoring= +2.5 Assess outcomes=+2.5 


Fluency Snapshots 


Not Using=0; Screening=+10 


Spelling inventories 


Not using=0; Screening=+2; Diagnostic = +2; Benchmarking=+2; 
Progress monitoring= +2 Assess outcomes=+2 


Score 


Sum of above Items/5 (Range from 0 to 10) 


Q9. Indicate extent you use student assessment data for 
each of the following purposes: 


a) Placing students in intervention programs; 


b) Differentiating instruction; 


c) Identifying skills that need to be re-taught; 


d) Monitoring student reading progress; 


e) Creating instructional groups 


Not at all=0; To a small extent=3.3; To a moderate extent=6.7; To a 
large extent=10 


Score 


Sum of above Items/5 (Range from 0 to 10) 
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Sub-Component 
13a: Data-driven 
instruction FOR 
TARGETED 
INTERVENTION 


Sub-Component 
13b: Data-driven 
instruction FOR 
INTENSIVE 
INTERVENTION 


Q10-11 Do you currently have grade-level teams at your 
school? If YES: Overall, rate the grade-level team’s ability 
to use classroom assessment data in the following ways: 


Address the literacy needs of all students 


Address the needs of struggling readers; 


Formalize lesson plans; 


No grade-level team= missing on all items; or Poor=0; Fair=3.3; 


Identify students who are eligible for targeted 
interventions; 


Identify strengths; 


Identify teaching and learning strategies 


Improve classroom practice 


Good=6.7; Excellent=10; Not sure=0 


Score 


Sum of above Items/7 (Range from 0 to 10) 


Total Sub-Component 12 Score 


Sum of the Above Scores/6 (Range from 0 to 10) 


LIST Surveys — from LIT Teacher (about the TARGETED intervention) 


Q11. Please indicate the extent to which you use student 
assessment data for each of the following purposes with 
students in the Targeted intervention group: 


e __ Differentiating instruction 


e Identifying skills that need to be retaught 


e Monitoring student reading progress 


e _ Creating instructional groups 


Not at all=0; To Some Extent=3.3; To a Moderate Extent=6.7; To a 
Large Extent=10 


Total Sub-component 13a Score 


Sum of the Above Items/4 (Range from 0 to 10) 


LIST Surveys — from LIT Teacher and AMP Teachers (about the intensive intervention) 


Q13. Please indicate the extent to which you use student 
assessment data for each of the following purposes within the 
afterschool program: 


e __ Differentiating instruction 


Identifying skills that need to be retaught 


Monitoring student reading progress 


Creating instructional groups 


Not at all=0; To Some Extent=3.3; To a Moderate Extent=6.7; To a 
Large Extent=10 


Total Sub-component 13b Score 


Sum of the Above Items/4 (Range from 0 to 10) 
Sum of the Above Scores/2 (Range from 0 to 10) 


Total Component 4 - Assessment Score: 


(Sum of above Sub-component Scores)/2 = Range from 0 to 10 


COMPONENT 5: HIGH QUALITY, HIGH INTEREST MATERIALS - Highly motivating reading materials integrated with engaging 
technology and audio resources. 


Components 


Individual/Summary Items 


Scores 


Sub-Component 
14: Text Sets 
(NOTE: Not 
included in the 
calculation of 
Component 5 
Score) 


Principal Interviews 


Q7. Are the school-wide text sets being used in the content area 
classrooms? 


e Social Studies 


e Science 


No or Not Used=0; Used=10; Don’t Know=Missing 


Total Sub-Component 14 Score (Text Sets) 


Sum of Above Scores/2 (Range from 0 to 10) 


Sub-Component 
15: School 
Library 


LIST Surveys — From ELA Teachers 


Q14a. For each of the materials listed below, indicate how 
frequently you currently use the materials to teach literacy: 
School Library 


N/A (Do Not Have)=0; Not Currently Using=0; Less than once a 


month=2.5; 1 to 3 times a month=5; | to 3 times a week=7.5; 4 to 5 


times a week=10 


Score 


Range from 0 to 10 


Q14b Rate you comfort level: School Library 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 4=7.5; 5 Very 
comfortable=10 


Score 


Range from 0 to 10 


Q29. How often do you take your class to the library? 


Never=0; Rarely=2.5; Sometimes=5; Often=7.5; Almost daily or 
daily=10 


Score 


Range from 0 to 10 


Q32. How does the librarian work with you? 


a) Does not work with me=0; If checked b) or c)=5; If checked b) and 


c)=10 
(Also, if no librarian (FS260), then =0) 


Score 


Range from 0 to 10 


Q33. To what extent does the librarian consult with classroom 
teachers in using Striving Readers library funds to order reading 
materials that are grade level and content appropriate? 


Not at all=0; To a small extent=3.3; To a moderate extent=6.7; To a 


large extent=10; Don’t know=0 


Score 


Range from 0 to 10 


Q34. How does the librarian work with your students? (Check 
all that apply.) 


e Does not work with my students. 


e Works with students on research skills. 


“Does not work with my students” or "no librarian"= 0 Otherwise: 


give 2.5 points per check with a maximum of 10 points 
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e __ Directs students to resources tied to curriculum. 


e Conducts read-alouds. 


e Provides students with information about 
extracurricular academic activities (e.g., spelling bee, writing 
competitions, events). 


e __ Assists students with class projects. 


e _ Teaches students how to navigate Internet resources. 


e Guides struggling readers to summer programs. 


Score 


Range from 0 to 10 


LIST Surv 


eys — From LITs 


Q12a. For each of the materials listed below, indicate how 
frequently you currently use the materials to teach literacy: 


N/A (Do Not Have)=0; Not Currently Using=0; Less than once a 
month=2.5; 1 to 3 times a month=5; | to 3 times a week=7.5; 4 to 5 


° School Library 


times a week=10 


Q12b Rate you comfort level: 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 4=7.5; 5 Very 


° School Library 


comfortable=10 


Score 


Sum of Above Items/2 (Range from 0 to 10) 


Total Sub-Component 15 Score (School Library) 


Sum of Above Scores/7 (Range from 0 to 10) 


Sub-Component 
16: Classroom 
Library 


LIST Surveys — From ELA Teachers 


Q14a. For each of the materials listed below, indicate how 
frequently you currently use the materials to teach literacy: 
Classroom Library 


N/A (Do Not Have)=0; Not Currently Using=0; Less than once a 
month=2.5; | to 3 times a month=5; | to 3 times a week=7.5; 4 to 5 
times a week=10 


Score 


Range from 0 to 10 


Q14b Rate you comfort level: Classroom Library 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 4=7.5; 5 Very 
comfortable=10 


Score 


Range from 0 to 10 


QIS5. For each of the materials listed, please indicate which 
literacy instructional goals are supported by your use of that 
material in your classroom: Classroom Library 


Not Using = 0; Checked=10 


e Vocabulary development 


e Fluency 


e Reading Comprehension 


e Word Recognition 


e To develop students’ self-directed learning 
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e To supplement students’ textbook reading 


e _To activate students’ prior knowledge 


Score 


Sum of Above Items/7 (Range from 0 to 10) 


Q23. Please check the ways that you use your classroom 
libraries: 


e For content area instruction 


e For independent reading 


e For small group instruction 


e _- For read alouds 


For each item: Not checked=0; Checked=10 


Score 


Sum of Above Items/4 (Range from 0 to 10) 


Q24. Do you use interest inventories to help students self 
select reading materials? 


No=0; Yes=10 


Score 


Range from 0 to 10 


Q25. Do you use interest inventories to guide your 
purchases for the classroom library? 


No=0; Yes=10 


Score 


Range from 0 to 10 


Q30 My classroom library... 


e _Iseasily accessible to students 


e _Is well organized and in good shape 


e Has acheckout system in place 


e __Includes a variety of reading materials 


e _Includes a variety of texts 


e __Has reading materials grouped by genre 


e __Has reading materials clearly labeled 


e _ Has both non-fiction and fiction books 


Not at all true=0; Slightly true=3.3; Somewhat true=6.7; Very true=10 


Score 


Sum of the Above Items/8 (Range from 0 to 10) 


Q27. To what extent are you able to consider students’ needs 
and reading abilities when ordering books and other reading 
material with Striving Readers library funds for your classroom 
library? 


Not at all=0; To a small extent=3.3; To a moderate extent=6.7; To a 
large extent=10; Don’t know=missing 


Score 


Range from 0 to 10 


Q28. To what extent are you able to consider students’ 
interests and motivation when ordering books and other 
reading material with Striving Readers library funds for your 
classroom library? 


Not at all=0; To a small extent=3.3; To a moderate extent=6.7; To a 
large extent=10; Don’t know=missing 
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Sub-Component 
17: Other Non- 
Technology 
Resources/Materi 
als 


Score 


Range from 0 to 10 


LIST Surveys — From LITs 


Q12a. For each of the materials listed below, indicate how 
frequently you currently use the materials to teach literacy: 


N/A (Do Not Have)=0; Not Currently Using=0; Less than once a 
month=2.5; 1 to 3 times a month=5; | to 3 times a week=7.5; 4 to 5 


° Classroom Library times a week=10 
Q12b Rate you comfort level: N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 4=7.5; 5 Very 
e Classroom Library comfortable=10 


Score 


Sum of Above Items/2 (Range from 0 to 10) 


Total Sub-Component 16 Score (Classroom Libraries) 


Sum of the Above Scores/9 (Range from 0 to 10) 


LIST Surveys — From ELA Teachers 


Q14a. For each of the materials listed below, indicate how 
frequently you currently use the materials to teach literacy: 


e Vocabulary Notebooks 


N/A (Do Not Have)=0; Not Currently Using=0; Less than once a 


e __ Reading response notebooks 


month=2.5; | to 3 times a month=5; | to 3 times a week=7.5; 4 to 5 


e Other informational text sets 


times a week=10 


Q14b Rate you comfort level: 


e Vocabulary Notebooks 


e __ Reading response notebooks 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 4=7.5; 5 Very 
comfortable=10 


e Other informational text sets 


Q15. For each of the materials listed, please indicate which 
literacy instructional goals are supported by your use of that 
material in your classroom: Vocabulary Notebooks 


e Vocabulary development 


e Word parts 


e Spelling 


° Grammar 


Not Using = 0; Checked=10 


e To develop students’ self-directed learning 


e _To activate students’ prior knowledge 


Score 


Sum of Above Items/7 (Range from 0 to 10) 


LIST Surv 


eys — From LITs 


Q12a. For each of the materials listed below, indicate how 
frequently you currently use the materials to teach literacy: 


N/A (Do Not Have)=0; Not Currently Using=0; Less than once a 
month=2.5; | to 3 times a month=5; | to 3 times a week=7.5; 4 to 5 


e Vocabulary Notebooks 


times a week=10 


e Reading ANTHOLOGIES 


N/A (Do Not Have)=0; Not Currently Using=0; Less than once a | 
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e Reading BASALS 


e Other informational texts (other than text sets) 


month=2.5; 1 to 3 times a month=5; | to 3 times a week=7.5; 4 to 5 


times a week=10 


Q12b Rate you comfort level: 


e Vocabulary Notebooks 


Not at all comfortable=0; 2=2.5; 3=5; 4=7.5; 5 Very comfortable=10 


e Reading ANTHOLOGIES 


e Reading BASALS 


e Other informational texts (other than text sets) 


Not at all comfortable=0; 2=2.5; 3=5; 4=7.5; 5 Very comfortable=10 


Score 


Sum of Above Items/8 (Range from 0 to 10) 


Total Sub-component 17 Score (Other Non-Technology 
Materials) 


Sum of the Above Scores/2 (Range from 0 to 10) 


Sub-Component 
18: Use of HHC 
for reading 
instruction as part 
of the targeted 
intervention 
model 


LIST Survey - from LIT only (Section about targeted intervention) 


Q14. Ina typical classroom, how often do your students use 
handheld computers (Palm Pilots) during Targeted Intervention 
instruction of Tier 2 and 3 students? 


Don’t use=0 (from Q8); Less than once a month=3.3; 1-3 times a 
month=6.7; 1-3 times a week=10; 4-5 times a week=10 


Score 


Range from 0 to 10 


QI5. Rate how comfortable you are with using the Palm Pilots 
to support your literacy instruction 


Don’t use or Not at all comfortable=0; 2=3.3; 3=6.7; 4=10; 5 Very 
comfortable=10 


Score 


Range from 0 to 10 


Q16. Which specific academic foci or instructional objectives 
do you support with the use of handheld computers (Palm 
Pilots) during Targeted Intervention instruction? (Check all that 


apply) 


e Fluency 


e Vocabulary development 


e Developing students’ reading comprehension 
strategies 


e Writing skills 


e Word parts 


e Word recognition 


© Spelling 


° Grammar 


e Organizing information 


e _ Locating information 


e Evaluating information 


e _Synthesizing information 


e Demonstrate knowledge of key concepts 


Not checked=0; 
Checked=10 


e To develop students’ self-directed learning 


e Teaching students to identify and use the 
organizational features of expository writing 


e _To activate students’ prior knowledge 


Score 


Sum of Above Items/16 (Range from 0 to 10) 


Q17. Which instructional methods do you support with the use 
of handheld computers (Palm Pilots) during the Targeted 
Intervention instruction? (Check all that apply) 


e __ Assessing students’ literacy skills 


e Monitoring students’ progress 


e Teaching comprehension skills 


e Using comprehension techniques 


e Guided reading 


e __ Partner reading 


e _ Individual reading 


e __ Book club disussions 


Not checked=0; 
Checked=10 


Score 


Sum of Above Items/8 (Range from 0 to 10) 


Q18. Please indicate whether you use handheld computers 
(Palm Pilots) to support grouping structures and/or 
differentiated instruction during Targeted Intervention 
instruction of Tier 2 and 3 students in the regular classroom, by 
checking the appropriate strategies below. (Check all that 


apply) 


Whole class/ Large group 


Small group/ Pairs 


Individual Work 


Differentiating instruction for struggling readers 


Differentiating instruction for English language 
learners/special education students 


Not checked=0; 
Checked=10 


Score 


Sum of Above Item/5 (Range from 0 to 10) 


Total Sub-component 18 Score (Handheld Computers for 
Targeted Intervention) 


Sum of the Above Items/5 (Range from 0 to 10) 


Sub-Component 
19: Other 
technology 


LIST Surveys — From ELA Teachers 


Q14a. For each of the materials listed below, indicate how 
frequently you currently use the materials to teach literacy: 


N/A (Do Not Have)=0; Not Currently Using=0; Less than once a 


month=2.5; 1 to 3 times a month=5; | to 3 times a week=7.5; 4 to 5 
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resources 


e __ Listening Centers 


times a week=10 


Q14b Rate you comfort level: 


e Listening Centers 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 4=7.5; 5 Very 
comfortable=10 


Score 


Sum of Above Items/2 (Range from 0 to 10) 


Q15. For each of the materials listed, please indicate which 
literacy instructional goals are supported by your use of that 
material in your classroom: Listening Centers 


e Vocabulary development 


e Fluency 


e Reading Comprehension 


e To develop students’ self-directed learning 


e To supplement students’ textbook reading 


e _To activate students’ prior knowledge 


Not Using = 0; Checked=10 


Score 


Sum of Above Items/6 (Range from 0 to 10) 


Principal Interviews 


Q12. For each of the following technology resources that your 
teachers are using to teach literacy, please indicate to what 
extent that technology resource is integrated into the literacy 
curriculum. 


e Listening Centers 


N/A or Not at all integrated=0; Somewhat integrated=5; Thoroughly 
integrated=10 


Score 


Range from 0 to 10 


Total Sub-sub-component 19a Score (Listening Centers) 


Sum of the Above Items/3 (Range from 0 to 10) 


LIST Surveys — From ELA Teachers 


Q14a. For each of the materials listed below, indicate how 
frequently you currently use the materials to teach literacy: 


e Media Centers 


N/A (Do Not Have)=0; Not Currently Using=0; Less than once a 
month=2.5; 1 to 3 times a month=5; | to 3 times a week=7.5; 4 to 5 
times a week=10 


Q14b Rate you comfort level: 


e Media Centers 


N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 4=7.5; 5 Very 
comfortable=10 


Score 


Sum of Above Items/2 (Range from 0 to 10) 


Q15. For each of the materials listed, please indicate which 
literacy instructional goals are supported by your use of that 
material in your classroom: Media Centers 


e Vocabulary development 


e Fluency 


e Reading Comprehension 


Not Using = 0; Checked=10 
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e Writing skills 


e _Word Parts 
e Word Recognition 
e Spelling 


e Grammar 


e To develop students’ self-directed learning 


e To supplement students’ textbook reading 


e Teaching students to identify and use the 
organizational features of expository writing 


e _To activate students’ prior knowledge 


Score 


Sum of Above Items/12 (Range from 0 to 10) 


Princip 


al Interviews 


Q12. For each of the following technology resources that your 
teachers are using to teach literacy, please indicate to what 
extent that technology resource is integrated into the literacy 
curriculum. 


N/A or Not at all integrated=0; Somewhat integrated=5; Thoroughly 
integrated=10 


° Media Centers 


Score 


Range from 0 to 10 


LIST Surv 


eys — From LITs 


Q12a. For each of the materials listed below, indicate how 
frequently you currently use the materials to teach literacy: 


N/A (Do Not Have)=0; Not Currently Using=0; Less than once a 
month=2.5; 1 to 3 times a month=5; | to 3 times a week=7.5; 4 to 5 


e Media Centers times a week=10 

Q12b Rate you comfort level: N/A=0; Not at all comfortable=0; 2=2.5; 3=5; 4=7.5; 5 Very 
e Media Centers comfortable=10 

Score Sum of Above Items/2 (Range from 0 to 10) 


Total Sub-sub-component 19b Score (Media Centers) 


Sum of the Above Items/4 (Range from 0 to 10) 


Total Sub-Component 19 Score (Listening Centers and 
Media Centers) 


Sum of the Above Sub-sub-Component Scores/2 (Range from 0 to 
10) 


Total Component 5 Score (Materials) 


(Sum of above Sub-component Scores)/5 = Range from 0 to 10 
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COMPONENT 6: CONTENT AREA INSTRUCTION - Integrating reading instruction and strategies into content area 
instruction (science, mathematics, and social studies) 


Components Individual/Summary Items Scores 

Subcomponent 20: | LIST Survey - from CAT only 

Literacy Q1. When, if at all, did you start integrating literacy into your This year = 10 , Last school year = 10 , Before Striving Readers 
Integration content area instruction? began = 10, I do not integrate literacy instruction into my content 


area. (4) =0 


Total Subcomponent 20 Score (Literacy Integration) 


Range from 0 to 10 


Subcomponent 21: 
Comprehension 


Q2. How often do you use the following practices to help 
students increase reading comprehension? 


Explicit instruction in the use of any one or more of the 
following comprehension strategies: summarizing, 
questioning, predicting, text structure, visualization, 
inferring and metacognition 


Never (1)=0; Less than once a month=3.3; 1-3 times a month=6.7; 1- 
3 times a week=10; 4-5 times a week=10; 


Comprehension Part I Score 


Range from 0 to 10 


Q2. How often do you use the following practices to help 
students increase reading comprehension? 


- Establishing the purpose for reading. 


- Monitoring students’ comprehension through 
questioning. 


- Use of before, during, and after (BDA) reading 
strategies for comprehension instruction 


Never (1)=0; Less than once a month=2.5; 1-3 times a month=5; 1-3 
times a week=7.5; 4-5 times a week=10; 


Q2. How often do you use the following practices to help 
students increase reading comprehension? 


- Making connections to background knowledge. 


- Making connections between texts. 


- Synthesizing information within text or across texts. 


Never (1)=0; Less than once a month=3.3; 1-3 times a month=6.7; 1- 
3 times a week=10; 4-5 times a week=10; 


Comprehension Part II Score 


Range from 0 to 10 


Q4. How often do you use the following techniques to help 
struggling readers develop better reading strategies and skills? 


Everybody Reads To (ERT) 


Exclusion Brainstorming 


List-Group-Label 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 
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; Interactive Notation System for Effective Reading 
and Thinking (INSERT) 


Guided Reading and Summarizing Procedure 


(GRASP) 
: ReQuest 
Predict-Locate-Add-Note (PLAN) . 
; Never/ Not Familiar=0; Less than once a month=5; 1-3 times a 
ABC Graffiti . : 
KWL month=10; 1-3 times a week=10; 4-5 times a week=10 


Comprehension Part III Score 


Sum of the Above Items/9 (Range from 0 to 10) 


Total Subcomponent 21 score (Comprehension) 


Sum of the Above Items/3 (Range from 0 to 10) 


Subcomponent 
2:Vocabulary 


Q3. How often do you use the following practices to help 
students build their vocabulary knowledge? 


Explicit instruction in vocabulary 


Modeling the use of word parts 


Review of vocabulary words 


Use of vocabulary notebooks 


: Use of before, during, and after (BDA) reading 
strategies for vocabulary instruction 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 


Academic Vocabulary for content terms 


Morphology instruction 


Word study sorts and concepts 


Words Their Way 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 


Total Subcomponent 22 Score (Vocabulary) 


Sum of the Above Items/9 (Range from 0 to 10) 


Subcomponent 23: 
PRC2 


Q2. How often do you use the following practices to help 
students increase reading comprehension? 


Using Pairing for Partner Reading & Content Too 
(PRC2) for comprehension instruction 


Q2. How often do you use the following practices to help 
students increase reading comprehension? 


Using differentiated instruction 


Q3. How often do you use the following practices to help 
students build their vocabulary knowledge? 


PRC2 for vocabulary development. 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 


Total Subcomponent 23 Score (PRC2) 


Sum of the Above Items/3 (Range from 0 to 10) 


1-27 


Subcomponent 24: 


Interactions with 
literacy experts 


QS. Please indicate how often (if at all) you discussed each 
of the following topics with any literacy experts during the 
current school year. 


Differentiated instruction 


Student groupings 


Use of Striving Readers text sets and text sets teacher 
guides. 


Use of technology resources: desktop computers, 
handhelds- palms, LCD projector, etc. 


Use of PRC2 instructional framework 


Using specific Striving Readers instructional techniques for 
comprehension instruction 


Using specific Striving Readers instructional techniques for 
vocabulary instruction 


Specific students’ reading progress. 


Using student assessment data for instructional planning 


Never=0; Less than once a month=3.3; 1-3 times a month=6.7; 1-3 
times a week=10; 4-5 times a week=10; 


Total Subcomponent 24 Score (Interactions with literacy 
experts) 


Sum of the Above Items/9 (Range from 0 to 10) 


Subcomponent 25: 


Use of Striving 
Readers Text Sets 
j and Text Sets 
Teacher Guides 


Q6. Do you use Striving Readers text sets to teach literacy 
through your content area? 


Yes = 10, No=0 


Text Set Part I Score 


Range from 0 to 10 


Q9. Rate how comfortable you are with using the Striving 
Readers text sets to support student learning in language arts. 


Not at all comfortable (1)=0; 2=2.5; 3=5; 4=7.5; 5 Very 
comfortable=10 


Q10. In your opinion, how well are the Striving Readers text 
sets aligned to the curriculum of your subject area? 


Poorly Aligned (1)=0; 2=2.5; Adequately Aligned (3)=5; 4=7.5; 5 
Very Well Aligned (5)=10 


Text Set Part II Score 


Sum of the Above Items/2 (Range from 0 to 10) 


Total Subcomponent 25 Score (Text Sets) 


Sum of the Above Items/2 (Range from 0 to 10) 


Total Component 6 Score (Literacy Integration) 


(Sum of above Sub-component Scores)/6 = Range from 0 to 10 
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COMPONENT 7: Integrated, progressive, high quality professional development 


Components 


Individual/Summary Items 


Scores 


Sub-component 
25: Whole School 
Professional 
Development 


Professional Development Attendance Records 


The percents below refer to the percent of meetings attended by principals. 


Principals’ Monthly Professional Development 


[<30%]=0; [30-59% ]=1; [60-79% ]=2; [80-100%]=3 


The percents below refer to the session attendance rates averaged across teachers 


Teachers’ Summer Institute (Yearly) 


[<25%]=0; [25-50% ]=1; [51-79%]=2; [80-100%]3 


Teachers’ Quarterly Follow-Up Institutes 


[<25%]=0; [25-50% ]=1; [51-79%]=2; [80-100%]F3 


Total Sub-Component 21a Score- Professional Development 
Attendance 


(Sum of the Above Items/9)*10 


LIST Survey - from ELA Teachers only 


Q39b. For each of the following Striving Readers professional 
development sessions conducted during the 2008-2009 school 
year, please indicate: how useful the session(s) was (were) in 
helping you support student learning in language arts. 


e 2009 Summer institute 


e —School-year follow-up institutes 


e Technology training (use of handhelds) 


e School-based professional development 


Not Participated=missing; Not useful=0; Somewhat useful=3.3; 
Moderately Useful=6.7; Extremely Useful=10 


Q40b. For each of the following topics, rate the impact that 
professional development you received has had on your comfort 
with each teaching practice. 


e Building academic vocabulary 


e Classroom libraries 


e Creating literacy-rich classroom environments 


e _ Differentiating instruction 


e Explicit vocabulary instruction 


e Increasing student motivation 


e Supporting students’ self-directed learning 


e Using before, during, and after reading strategies 


e Using student assessments to guide instruction 


e — Using handheld computers (Palm Pilots) 


e _ Using literacy-based software 


Not Participated=missing; No Impact=0; Some Impact=3.3; 
Moderate Impact=6.7; Large Impact=10 
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e Using the PRC2 model 


e —_ Using the whole-part-whole classroom instruction 
model 


Score 


Sum of the Above Items (Range from 0 to 10)/N of items with 
available data 


Principal Interviews 


QIS5. For each of the following Striving Readers professional 
development sessions conducted during the 2008-2009 school 
year, please indicate: how useful the session(s) was (were) in 
helping you support student learning in language arts. 


e Monthly Principal Meetings (Leaders Seminars) 


° 2009 Summer institute 


e _School-year follow-up institutes 


e Saturday seminars 


e On-site training during literacy team meetings 


e School-based Striving Readers professional 
development 


Not Participated=missing; Not useful=0; Somewhat useful=3.3; 
Moderately Useful=6.7; Extremely Useful=10 


Total Score For Principals 


Sum of the Above Items (Range from 0 to 10)/6 


Total Sub-Component 21b Score - Professional Development 
Survey Responses 


(Sum of the Above Items)/2 = Range from 0 to 10 


Total Sub-Component 25 Score 


Sum of Above Scores/2 (Range from 0 to 10) 


Sub-Component 
26: Targeted and 
Intensive 
Intervention 


LIS Survey - from LITs 


Q21b. For each of the following Striving Readers professional 
development sessions conducted during the 2009-10 school 
year, please indicate: how useful the session(s) was (were) in 
helping you support student learning in language arts. 


e AMP Intensive Intervention Program Training 


° 2009 Summer institute 


e _School-year follow-up institutes 


e _ Bi-weekly LIT training sessions 


° Teacher/LIT collaboration 


e School-based professional development 


Not Participated=missing; Not useful=0; Somewhat useful=3.3; 
Moderately Useful=6.7; Extremely Useful=10 


Score 


Sum of the Above Items (Range from 0 to 10)/6 of items with 
available data 
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Q22b. For each of the following topics, rate the impact that 
professional development you received has had on your comfort 
with each teaching practice. 


e Building academic vocabulary 


e Classroom libraries 


e Creating literacy-rich classroom environments 


e _ Differentiating instruction 


e Explicit vocabulary instruction 


e Increasing student motivation 


e Supporting students’ self-directed learning 


e Using before, during, and after reading strategies 


e Using student assessments to guide instruction 


e Using handheld computers (Palm Pilots) 


e _ Using literacy-based software 


e Using the PRC2 model 


Not Participated=missing; No Impact=0; Some Impact=3.3; 
Moderate Impact=6.7; Large Impact=10 


e Training in LIT/ teacher collaboration 


Not Participated=missing; Not useful=0; Somewhat useful=3.3; 
Moderately Useful=6.7; Extremely Useful=10 


Score 


Sum of the Above Items (Range from 0 to 10)/13 of items with 
available data 


Professional Development Attendance Records 


The percents below refer to the percent of meetings attended by LIT or principals, respectively. 


LIT Weekly Meetings with Coordinators 


[<60%]=0; [60-74%]=2.5; [75-89%]=5; [90-100%]=10 


Score 


Range from 0 to 10 


Total Sub-Component 26 Score 


Sum of Above Scores/3 (Range from 0 to 10) 


Total Component 7 Score - Professional Development 


(Sum of above sub-components)/2 (Range from 0 to 10) 
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Appendix J: Year 4 Fidelity Scale Results by School 


Table J-1 
Year 4 Implementation Fidelity Scales by School: Major Program Components 
Mean Score 
Component Component 
era ae Overall rae ey tine ocae a Component 6 Caen 
pide Blended Targeted Intensive Date Daven : : Content area Professional 
Score : : . _ |Instruction &| Materials Literacy 
Intervention | Intervention | Intervention E Development 
Assessment Instruction 

4 7.8 6.0 8.9 8.9 8.3 71 72 8.9 

5 73 7.0 8.1 7.2 7.8 6.4 7.2 8.3 

6 71 6.8 7.0 79 Ted 6.4 71 6.9 

8 7A 7.6 8.8 7.6 5.6 7.2 7.7 8.4 

11 6.7 1 4.1 7.7 6.7 6.8 6.5 6.8 

13 = 6.9 9.2 8.1 a 5.5 8.1 7.8 

16 7.8 7.6 7.9 7.0 8.3 8.1 73 8.3 

Cohen 17 8.8 8.4 9.3 8.3 9.3 8.6 9.2 8.2 

1 19 ‘Tel 6.7 8.7 7.8 8.2 71 73 8.1 

20 6.6 75 6.1 6.7 5.6 7.2 7A 7A 

22 8.1 7.8 8.7 8.3 8.7 7.0 7.9 7A 

24 8.0 9.0 9.3 74 6.9 6.0 9.3 6.0 

27 8.0 12 8.6 8.8 8.7 6.9 7A 8.2 

29 7.2 7.0 8.4 7.9 7.2 5.6 7A 71 

30 8.1 6.5 9.2 9.0 9.3 6.2 8.5 75 

31 7.6 7.2 8.8 8.4 7.0 6.5 6.8 9.1 

Total 7.6 7.3 8.2 7.9 7.7 6.9 7.6 7.8 

1° = ee as ee = 22 a = 

2 8.4 7.9 9.2 8.3 8.9 Te 9.1 8.1 

34 ne = 7 ze ne <3 7 a3 

7 8.1 8.2 8.3 8.1 8.2 7.8 9.2 6.6 

9 ‘hed, 6.8 7.8 8.1 75 5.8 8.8 7.5 

10 = 7.5 8.4 1.7 ey 4.6 8.3 6.0 

12 7.9 8.1 8.4 7A 8.2 7.2 7.6 6.5 

Cohort 14 7.2 6.3 8.4 8.2 7.2 6.0 10.0 8.0 

2 15 8.3 8.0 9.3 8.3 8.7 6.9 9.0 7.7 

16 7A 7.0 8.0 71 8.0 6.9 8.9 71 

21 73 7.6 7.0 7.7 75 6.6 8.2 7.9 

23 7A 74 8.8 79 6.2 6.6 8.0 6.1 

25 = --° 8.2 6.9 -- -- 5.9 -- 

26 6.6 7.8 7.8 73 4.9 5.1 8.0 7.3 

28 ane = 7.0 8.5 -- -- 6.4 6.3 

Total 7.6 75 8.2 78 75 6.5 8.3 all 

Overall 7.6 7.4 8.2 7.9 7.6 6.7 7.9 7.4 


“The overall fidelity score is based on components | through 5. 
> The principal interview could not be completed for this school; score could not be calculated. 


“This school closed after SY 2008-09. 


4 This school became a turn-around school as of SY 2009-10. 
“No ELA teacher surveys received for this school; score could not be calculated. 
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Component 1: Blended intervention 


Table J-2 
Results of Year 4 implementation fidelity scales by school 


Mean Score 
Component d pap gute oee a Seas caarE ses Eee 
rate) | bane Use of PRC2, 
Number text sets and 
Blended Small Group Comprehension technology to Marzano’s 
Intervention Instruction Focus support Vocabulary 
differentiated 
instruction 

4 6.0 5.6 73 5.3 Ds 
5 7.0 7.3 8.3 6.9 5.6 
6 6.8 7.0 73 7.3 5.7 
8 7.6 6.7 8.2 8.0 75 
11 75 74 8.0 7.6 ha 
13 6.9 6.3 8.0 7.2 6.1 
16 7.6 7.7 8.9 73 6.4 
17 8.4 7.9 9.0 9.1 7.7 
Cohort | 19 6.7 cel 74 6.2 6.2 
20 ie) 6.8 8.2 7.6 7.6 
22 7.8 74 8.1 8.1 7.6 
24 9.0 9.2 8.8 9.5 8.4 
27 ho 6.9 8.5 7.1 6.3 
29 7.0 7.3 8.0 6.5 6.1 
30 6.5 7.0 7.3 5.9 6.0 
31 be 6.6 8.2 6.9 6.9 
Cohort 1 es 71 8.1 73 6.7 
1 a -- im =z as 
2 7.9 7.2 8.1 8.1 8.2 
3 a 2s ae oh mp 
7 8.2 8.3 9.0 7.6 7.8 
9 6.8 6.7 7.9 6.6 6.0 
10 7.5 7.7 8.0 6.9 75 
12 8.1 7.9 8.4 8.2 8.0 
Cohort 2 14 6.3 6.8 BS 4.4 6.5 
15 8.0 7.3 8.9 7.6 8.4 
16 7.0 6.3 8.1 73 6.3 
21 7.6 7.9 8.8 74 6.4 
23 74 7.0 6.8 Teal 8.2 
25 -- -- -- -- -- 
26 7.8 8.2 8.3 6.3 8.4 
28 -- -- -- -- -- 
Cohort 2 75 74 8.2 Toll 7.4 
Overall 7.4 7.2 8.1 7.2 
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Component 2: Targeted intervention 


Table J-3 
Results of Year 4 implementation fidelity scales by school 


Mean Score 
Sub- Sub- 
Component 2 Component Component 

Ganort School 5 6 

Number Direct instruction 

Targeted Teacher/LIT | in comprehension, 

Intervention Collaboration vocabulary and 
fluency 

4 8.9 9.3 8.5 
5 8.1 7.7 8.6 
6 7.0 6.8 7.2 
8 8.8 9.6 8.0 
11 4.1 3.3 4.9 
13 9.2 10.0 8.3 
16 7.9 7.0 8.7 
17 9.3 9.6 9.0 
re 19 8.7 9.0 8.3 
20 6.1 6.8 5.4 
22 8.7 8.1 9.2 
24 9.3 9.4 9.1 
27 8.6 9.1 8.1 
29 8.4 8.6 8.2 
30 9.2 9.1 9.4 
31 8.8 9.4 8.2 
Cohort 1 8.2 8.3 8.1 
1 = a = 
2 9.2 9.9 8.5 
3 = a = 
7 8.3 8.8 7.8 
9 7.8 9.2 6.4 
10 8.4 9.2 7.6 
12 8.4 9.6 7.2 
Cohort 14 8.4 7.8 9.1 
2 15 93 9.6 9.0 
16 8.0 9.2 6.9 
21 7.0 6.6 7.3 
23 8.8 9.4 8.1 
25 8.2 10.0 6.3 
26 7.8 8.8 6.9 
28 7.0 7.8 6.2 
Cohort 2 8.2 8.9 75 
Overall 8.2 8.6 78 
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Table J-4 


Results of Year 4 implementation fidelity scales by school 
Component 3: Intensive Intervention 


Mean Score 
rs School Component3 Sub-Component Sub-Component Sub-Component Sub-Component SubsComponent 11 
ohort 7 8 9 10 
PUSS Increased Instructional Explicit Instruction in Explicit Instruction in Explicit Instruction in 
Intensive Intervention i Small Group Setting (15:1) 5 
Time Comprehension Vocabulary Fluency 

4 8.9 8.4 10.0 10.0 10.0 6.2 

5 7.2 75 10.0 6.7 4.8 6.9 

6 7.9 6.5 10.0 8.4 vii 6.8 

8 7.6 5.7 10.0 8.4 5:5 8.3 

11 7.7 6.9 10.0 6.7 7.1 7.9 

13 8.1 Td 10.0 6.7 7.5 9.1 

16 7.0 7.6 6.8 6.7 7.0 6.9 

17 8.3 8.2 10.0 8.4 74 74 

ota 19 78 5.9 10.0 8.4 6.8 7.9 
20 6.7 3.8 10.0 8.4 6.5 4.9 

22 8.3 6.1 10.0 8.4 8.0 9.1 

24 74 6.2 10.0 6.7 7.6 6.5 
27 8.8 6.5 10.0 10.0 7.5 10.0 

29 7.9 7.2 10.0 6.7 8.2 74 

30 9.0 8.9 10.0 8.4 9.0 8.6 

31 8.4 74 10.0 8.4 8.4 7.9 

Total 79 6.9 9.8 79 7.4 7.6 

1 = be ae 3 as a 

2 8.3 5.0 10.0 10.0 8.0 8.6 

3 = 2s oe aC a2 ba 

7 8.1 Lal: 10.0 8.4 6.6 7.9 

9 8.1 6.3 10.0 8.4 7.2 8.6 

10 7.7 5.8 10.0 8.4 6.8 74 

12 74 4.8 10.0 10.0 5.6 6.7 

Cohort 14 8.2 5.3 10.0 10.0 8.1 7.6 
2 15 8.3 7.0 10.0 8.4 7.9 8.2 
16 7A 4.8 10.0 8.4 4.9 7.3 

21 7.7 6.2 10.0 10.0 6.3 6.2 

23 7.9 6.4 10.0 8.4 7.8 6.9 

25 6.9 5.0 10.0 6.7 7.0 5.9 

26 7.3 5.9 10.0 5.0 6.9 8.6 

28 8.5 5.3 10.0 10.0 8.4 8.9 

Total 7.8 5.8 10.0 8.6 7.0 7.6 

Overall 79 6.3 9.9 8.3 7.2 7.6 
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Table J-5 
Results of Year 4 implementation fidelity scales by school 
Component 4: Purposeful Assessment & Data Driven Instruction 


Mean Score 
Component 4 Sub-Component Sub- 
Cohort peal 12 Component 13 
Number Purposeful : 
Neto RA TCH Whole School/Blended Targeted and Intensive 
Driven ictinion Intervention Intervention 
4 8.3 6.6 10.0 
5 7.8 7.8 7.9 
6 7.2 7.3 7.1 
8 5.6 5.0 6.3 
11 6.7 6.2 7.1 
13 -- -- 9.2 
16 8.3 8.6 7.9 
17 9.3 8.9 9.6 
a 19 8.2 8.0 8.4 
20 5.6 4.9 6.3 
22 8.7 7.5 10.0 
24 6.9 6.7 7.1 
27 8.7 7.9 9.6 
29 7.2 6.1 8.4 
30 9.3 8.7 10.0 
31 7.0 6.5 75 
Cohort 1 Vell 71 8.3 
1 _ = _ 
2 8.9 7.7 10.0 
3 Ss cA a 
7 8.2 8.8 75 
9 7.5 6.6 8.3 
10 _ -- 5.4 
12 8.2 8.4 7.9 
Cohort 14 72 4.5 10.0 
: 15 8.7 TA 10.0 
16 8.0 8.5 75 
21 75 6.7 8.4 
23 6.2 7.0 5.4 
25 =4 -- 4.6 
26 4.9 4.4 5.4 
a = - 67 
Cohort 2 75 7.0 75 
Overall 7.6 71 79 
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Table J-6 


Results of Year 4 implementation fidelity scales by school 
Component 5: Materials 


Mean Score 
Gaanecaltesparqinnrenae Component 5 Reema aan Seige ca Sub-Component 17 Sui a Sub-Component 19 
Materials Text sets School library Classroom library OSES panded Obes npology 
nology resources computers resources 
4 71 10.0 8.5 8.0 5.8 44 5.8 
5 6.4 10.0 6.1 8.6 2.8 4.7 6.4 
6 6.4 10.0 6.1 8.6 3.6 5.1 4.8 
8 7.2 10.0 6.1 8.8 6.9 6.2 5.3 
11 6.8 10.0 5.8 9.3 6.0 5.4 7.3 
13 5.5 -- 0.9 8.7 6.2 8.1 3.7 
16 8.1 10.0 Bee) 9.2 6.9 8.6 8.3 
17 8.6 10.0 8.9 9.5 7.7 71 8.3 
Cohort | 19 7A 10.0 4.4 8.7 5.8 6.2 74 
20 7.2 10.0 5.9 9.3 6.2 5.3 6.2 
22 7.0 10.0 3.8 8.2 6.9 5.3 7.9 
24 7.2 10.0 1.7 8.6 7.5 74 8.1 
27 6.9 10.0 3.5 9.2 5.0 7.8 5.7 
29 5.6 10.0 4.5 9.1 Dull 0.0 4.3 
30 6.2 10.0 5.2 8.6 6.1 3.6 3.8 
31 6.5 10.0 7.8 8.0 6.6 0.0 6.8 
Cohort 1 6.9 10.0 5.3 8.8 6.0 5.3 6.3 
1 = oe 23 se ae aS at 
2 7.7 10.0 5.0 9.3 7.9 6.7 7.5 
3 se as us de aa ae aa 
i 7.8 10.0 7.7 9.3 7.5 4.7 7.8 
9 5.8 5.0 4.6 8.3 6.0 6.3 4.8 
10 4.6 10.0 2.5 8.6 6.6 0.0 5.0 
12 L2 10.0 4.5 9.2 6.4 6.0 7.3 
14 6.0 5.0 74 9.1 8.6 0.0 5.6 
one? 15 6.9 10.0 2.3 93 78 4.2 78 
16 6.9 10.0 5.8 8.8 6.3 4.5 6.3 
21 6.6 10.0 4.7 7.9 6.7 3.7 6.5 
23 6.6 10.0 2.9 8.6 6.5 6.7 5.0 
25 -- -- -- -- -- 5.4 -- 
26 5.1 10.0 0.0 9.1 6.3 0.0 5.4 
28 -- -- -- -- -- 6.1 -- 
Cohort 2 6.5 9.1 4.3 8.9 7.0 4.2 6.3 
Overall 6.7 9.6 4.8 8.8 6.5 4.8 6.3 
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Table J-7 


Results of Year 4 implementation fidelity scales by school 
Component 6: Content Area Literacy Instruction 


Mean Score 
a School Component 6 Sub-Component 20 Sub-Component Sub-Component Sub-Component 23 Sub-Component Sub-Component 
ohort Ninnher 21 DD; 24 25 
Content ared Mere. Integration Comprehension Vocabulary PRC2 Interaction Text sets 
Instruction 
4 7.23 10.00 8.44 6.67 5.55 6.03 6.72 
5 7.22 10.00 6.31 4.74 5.79 7.48 9.00 
6 7.14 10.00 6.17 6.36 6.86 6.56 6.88 
8 7.71 10.00 7.33 7.67 6.66 8.46 6.15 
11 6.54 10.00 6.83 6.34 7.09 6.03 2.97 
13 8.07 10.00 8.13 8.16 7.78 7.91 6.46 
16 7.29 10.00 7.22 6.74 6.56 6.55 6.69 
17 9.24 10.00 8.58 8.61 9.73 9.91 8.59 
Cohort | 19 7.33 8.33 6.56 6.74 7.33 7.54 7.50 
20 7.44 10.00 7.36 6.81 7.78 6.91 5.78 
22 7.93 10.00 8.04 8.57 8.90 7.33 4.77 
24 9.29 10.00 9.61 8.52 8.89 9.75 8.96 
27 7.35 10.00 8.47 7.49 7.11 5.55 5.50 
29 7.37 10.00 6.79 7.63 7.33 5.74 6.75 
30 8.46 10.00 8.90 8.15 8.34 6.76 8.59 
31 6.76 10.00 7.60 6.67 6.77 5.47 4.03 
Cohort 1 7.65 9.90 7.65 7.24 7.40 712 6.58 
1 as za Ss se as =o ae 
2 9.10 10.00 9.32 10.00 10.00 9.27 6.04 
3 se a a2 ze 2 an aa 
7 9.19 10.00 8.84 8.80 9.45 8.99 9.06 
9 8.83 10.00 8.55 8.81 8.74 7.84 9.02 
10 8.32 10.00 7.87 8.02 7.78 7.91 8.33 
12 7.56 6.67 8.77 8.34 8.90 7.66 5.00 
Cc 14 10.00 10.00 10.00 10.00 10.00 10.00 10.00 
ohort 2 
15 9.05 10.00 9.06 9.19 8.46 8.46 9.13 
16 8.90 10.00 8.59 8.23 8.45 8.52 9.63 
21 8.23 10.00 8.47 8.40 6.68 6.95 8.91 
23 7.99 10.00 6.54 6.86 8.35 7.42 8.75 
25 5.76 10.00 6.07 6.67 3.33 8.51 .00 
26 7.95 10.00 7.58 9.23 8.06 8.90 3.91 
28 6.44 10.00 6.76 7Al 8.90 5.57 .00 
Cohort 2 8.26 9.74 8.19 8.46 8.24 8.15 6.75 
Overall 7.92 9.83 7.89 7.79 7.78 7.59 6.66 
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Table J-8 
Results of Year 4 implementation fidelity scales by school 
Component 7: Professional Development 


Mean Score 
tek: Component 7 Sub-Component Sub-Component 
Cohort 26 27 
Number : Whole ; 
Professional Targeted/Intensive 
Development pleat icnded Intervention 
Intervention 

4 8.9 7.9 9.9 

5 8.3 7.2 9.5 

6 6.9 7.9 5.9 
8 8.4 6.8 10.0 

11 6.8 4.7 8.9 

13 7.8 5.6 9.9 

16 8.3 8.0 8.6 

Cshort 17 8.2 7.0 9.4 
1 19 8.1 7.2 9.0 
20 7.3 6.1 8.6 

22 7.3 7.7 7.0 

24 6.0 4.2 7.9 

27 8.2 8.2 8.2 

29 7.1 5.7 8.4 

30 7.5 5.4 9.6 

31 9.1 8.4 9.9 

Cohort 1 7.8 6.5 8.8 

1 ae Je ee 

2 8.1 6.7 9.4 

3 a ee ee 

7 6.6 7.5 5.7 

9 7.5 7.1 7.9 

10 5.9 4.6 7.3 

12 6.5 5.9 7.2 

Cohort 14 8.0 6.5 9.5 
2 15 7.7 5.7 9.6 
16 7.1 8.0 6.2 

21 7.9 7.3 8.4 

23 6.1 6.0 6.3 

25 7.2 5.6 8.9 

26 7.3 7.1 7.4 

28 6.3 6.7 6.0 

Cohort 2 Toll 6.5 Tell 

Overall 7.4 6.6 8.2 
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Appendix K: Year 4 Principal Interviews, Selected 
Comparisons of Treatment and Control Schools 


Table la—Presence of Literacy Team 


ie 


Does your school have a Literacy Team (a “vertical” team focusing on literacy issues across grade levels)? 


Treatment Schools (N = 26) Control Schools (N = 23) 


88.5% 78.3% 
11.5% 21.7% 


* In this table there were no statistically significant differences (p<.05) in the distribution of responses between Treatment and 
Control groups based on a Chi-Square test. 


Table I1b—Composition of Literacy Team 


Treatment Schools (N = 23) Control Schools (N = 17) 


91.3% 100% 


Table |c—Frequency of Literacy Team Meetings 


How often does the Literacy Team meet? 


Treatment Schools (N = 22) 


An inferential eeacistical canna was not conducted for this question 
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Table 2—Quality of the Literacy Team’s Performance 


Overall, rate the quality of the literacy team’s performance in the following areas. 


Treatment Schools (N = 23) Control Schools (N = 18) 
Item* Poor Fair Good Excell et Poor Fair Good Excell Net 
ent Sure ent Sure 
Addressing the 
needs of all 0% 8.7% 43.5% 47.8% 0% 0% 11.1% 33.3% 55.6% 0% 
students. 
Addressing the 


needs of struggling 4.3% 13% 304% 52.2% 0% 5.9% 5.9% 23.5% 64.7% 0% 
readers. 

Addressing the 

needs of grade-level 0% 9.1% 40.9% 50% 0% 0% 5.6% 33.3% 55.6% 5.6% 
teams. 

Addressing the 

needs of individual 0% 13% 39.1% 47.8% 0% 0% 16.7% 38.9% 44.4% 0% 
teachers. 
Addressing school 
wide needs (grades 
6-8) included in 
SIPAAA. 

Using assessment 
data and or student 
work to drive 
instruction. 
Supporting vertical 
and horizontal 
teacher 
collaboration. 
Improving literacy 
instruction at your 0% 13% 30.4% 56.5% 0% 0% 11.1% 38.9% 444% 5.6% 
school. 

*In this table there were no statistically significant differences (p<.05) in the distribution of responses between Treatment and 
Control groups based on a Mann-Whitney U test. 


0% 8.7% 34.8% 56.5% 0% 0% 16.7% 27.8% 55.6% 0% 


4.3% 8.7% 34.8% 52.2% 0% 0% 11.1% §=61.1% 27.8% 0% 


0% 13% 39.1% 47.8% 0% 0% 11.1% 50% 38.9% 0% 


Table 3a—Presence of Grade Level Teams 


Does your school have Grade Level Teams (“horizontal” teams consisting of staff across subject areas from the 
same grade)?* 


Treatment Schools (N = 26) Control Schools (N = 23) 
Yes 92.3% 95.7% 
No 7.7% 4.3% 


*In this table there were no statistically significant differences (p<.05) in the distribution of responses between Treatment and 
Control groups based on a Chi-Square test. 


Table 3b>—Composition of Grade-Level Teams 


Which of your staff are members of grade-level teams? 


Treatment Schools (N = 24) Control Schools (N = 22) 


91.7% 77.3% 


An inferential statistical comparison was not conducted for this question 


Table 3c—Frequency of Grade Level Team Meetings 


How often do the grade level teams meet? 


0% 


0% 


An inferential statistical comparison was not conducted for this question 
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Table 4—Quality of the Grade Level Teams’ Performance 


Overall, rate the quality of the Grade Level Teams’ performance in the following areas. 


Treatment Schools (N = 24) Control Schools (N = 22) 
Item* Poor Fair Good Excell Not Poor Fair Good Excell Not 
ent Sure ent Sure 
Bedressie te 0% 16.7% 50% 33.3% 0% 0% 9.1% 45.5% 45.5% 0% 
needs of all students 
Addressing the 


needs of struggling 8.3% 12.5% 37.5% 41.7% 0% 4.5% 18.2% 54.5% 22.7% 0% 
readers. 
Using assessment 
data to plan 0% 20.8% 29.2% 50% 0% 0% 18.2% 63.6% 18.2% 0% 
instruction 
Using assessment 
data to establish 
vertical and 8.3% 4.2% 58.3% 29.2% 0% 0% 31.8% 45.5% 22.7% 0% 
horizontal literacy 
goals by grade level 

mprevnehiere 
instruction at your 0% 8.3% 37.5% 54.2% 0% 0% 9.1% 63.6% 22.7% 45% 
school 
*In this table there were no statistically significant differences (p<.05) in the distribution of responses between Treatment and 
Control groups based on a Mann-Whitney U test. 


Table 5—Extent School Uses Assessment Data for the Following Purposes 


To what extent is student assessment data used for this purpose? 


Treatment Schools (N = 26) Control Schools (N = 23) 
Toa Toa Toa Toa Toa Toa 
e Not Not 
Item oe small moderate large aba small moderate large 
extent extent extent extent extent extent 


Screening students’ ability 

levels for placement in 0% 3.8% 19.2% 76.9% 0% 4.3% 52.2% 43.5% 
intervention programs* 

Diagnosing students’ 

strengths and support 

needs for placement in 0% 3.8% 30.8% 65.4% 0% 8.7% 34.8% 56.5% 
specific courses or 
instructional groups 
Identifying trends in fluency 
and comprehension 
abilities across groups of 
students 

Identifying trends in 
vocabulary knowledge 3.8% 11.5% 34.6% 50% 4.5% 13.6% 50% 31.8% 
across groups of students 

Monitoring overall student 

progress for the purpose 

of assessing success of 0% 0% 42.3% 57.7% 4.3% 4.3% 34.8% 56.5% 
instructional programs and 

methods 


0% 0% 46.2% 53.8% 0% 26.1% 26.1% 47.8% 


Differentiating instruction 0% 7.7% 42.3% 50% 4.3% 21.7% 26.1% 47.8% 


Planning on-site 
professional development 
*An asterisk in this column denotes a statistically significant difference (p<.05) in the distribution of responses between 
Treatment and Control groups based on a Mann-Whitney U test. 


0% 0% 28.0% 72% 0% 8.7% 26.1% 65.2% 


Table 6—Extent Non-Literacy Teachers Integrate Literacy Into the Following 


Subjects 


To what extent do non-literacy teachers integrate literacy into the content areas? 


Treatment Schools (N = 26) 


Niotat Toa 
Item* al . small 
extent 
Math 0% 30.8% 
Social Studies 0% 0% 
Science 0% 11.5% 


Toa 
moderate 
extent 


38.5% 


26.9% 


30.8% 


large 
extent 


30.8% 
73.1% 


57.7% 


Control Schools (N = 23) 


Toa Toa Toa 
Not at 
all small moderat large 
extent | eextent extent 
0% 30.4% 34.8% 34.8% 
0% 0% 43.5% 56.5% 
0% 13% 39.1% 47.8% 


*In this table there were no statistically significant differences (p<.05) in the distribution of responses between Treatment and 


Control groups based on a Mann-Whitney U test. 


Table 7—Use of the Following Technology Resources for Literacy Instruction 


Use of Technology Resources 


Treatment Schools (N = 26) 


Resource* Yes 

| Media Centers* 100% 
Listening Centers 96.2% 
Handhelds/Laptops* 100% 


Control Schools (N = 23) 


Yes No 
69.6% 30.4% 
82.6% 17.4% 
61.9% 38.1% 


*An asterisk in this column denotes a statistically significant difference (p<.05) in the distribution of responses between 
Treatment and Control groups based on a Chi-Square test. 


Table 8—Extent the Following Technology Resources are Integrated Into Literacy 
Curricula 


To what extent that technology resource is integrated into the literacy curriculum. 


Treatment Schools (N = 26) Control Schools (N = 21, 21, 22) 
Resource’ negra | SUPE reseed [NA | incre | HERE | regan 
Media Centers* 3.8% 0% 38.5% 57.7% | 19% 0% 71.4% 9.5% 
Listening Centers* 0% 3.8% 57.7% 38.5% 19% 0% 71.4% 9.5% 
pees 0% 3.8% 46.2% 50% 19% 45% 50% 9.1% 


*An asterisk in this column denotes a statistically significant difference (p<.05) in the distribution of responses between 
Treatment and Control groups based on a Mann-Whitney U test. 


Table 9—Overall Integration of Technology into the Literacy Curriculum 


Overall, how well is technology integrated into the literacy curriculum? 


Treatment Schools (N = 22) Control Schools (N = 26) 
Item* Not at all Somewhat Thoroughly Not at all Somewhat Thoroughly 
integrated integrated integrated integrated integrated integrated 
ae 0% 46.2% 53.8% 0% 81.8% 18.2% 
Integration 


*An asterisk in this column denotes a statistically significant difference (p<.05) in the distribution of responses between 
Treatment and Control groups based on a Mann-Whitney U test. 


Appendix L: Annotated Literacy Improvement 
Surveys 


ELA Teacher Surveys, Content Area Teacher 
Surveys, Literacy Intervention Teacher Surveys 


CPS Striving Readers Evaluation: 
Spring 2010 Literacy Improvement Survey Results 


All Treatment Schools: Content Area Teachers* 


*Content Area Teachers are defined as those teachers who provide instruction in mathematics, science and/or social 
studies. These may include those who teach literacy or common branch teachers, as well as special education and/or 
ELL teachers. 


&Q metis associates 


120 Wall St 
21" Floor 
New York, NY 10005 
212-425-8833 


Survey Response Rates: 


Respondents N Rate + 
pAbbott* | | - NA 
[Beethoven {| 6 | 3 50.0% 
|Bethune** [| 4 | 0.0% 
[Burroughs {| 8 | S| 62.5% 
[Carson | 9  6.7% 
Coles | 6 6.7% 
}Cook | 77 100.0% 
PDett 8 100.0% 
[Eberhart | 19 | 8 42% 
[Fiske | 4 | 875.0% 
[Fuller | 4 | 375.0% 
PGale |S 20.0% 
|Gompers_ | 8 |S 62.5% 


Henson 5 100.0% 


Linne | 6 
5 
| 8 

2 
3 
1 
4 
5 
1 
5 
4 


nN 
SIn 
& 
x 


Lovett 0.0% 
Manierre 50.0% 
Marsh 80.0% 
McCorkle 


50.0% 
75.0% 
Price 

Reavis 
Salazar 
Smyth 
Talcott 
Telpochcalli 
Volta 


100.0% 
125.0% 
10.0% 


| S| 83.3% 
| 4 


100.0% 


11 100.0% 


i a 
fe _ 
1 | 25.0% 
| 4 
p= 25s | 
i ae 


4 


° 
ge) 
oO 
= — 


+ Response rates exceed 100% at some schools where surveys were received from more content 
area teachers than were indicated in the district's records. 


* Abbott, one of the original Striving Readers schools, was not included in this survey 
administration because it closed after SY09. 


** No responses were received from Bethune because it had been identified as a turnaround 
school as of the 2009-2010 School Year. In order to reflect the original random selection of 
participating schools, however, it is still included in the calculation of response rates. 


When, if at all, did you start integrating literacy into your content area instruction? 


Before I do not integrate literacy 
Striving instruction into my content 
This year Readers began area 


N % % % % 
Piss | 96% | 26.7% | 622% 


Comprehensive Instruction 


In a typical classroom, how often do you use the following teaching practices to help 
students increase reading comprehension? 


Explicit instruction in the use of one or more of the 
following comprehension strategies: summarizing, 
questioning, predicting, text structure, visualization, 
inferring and metacognition 

Establishing the purpose for reading 

Monitoring students' comprehension through 
questioning 

Making connections to background knowledge 


Making connections between texts 


Synthesizing information within text or across texts 


Using differentiated instruction 


Use of before, during, and after (BDA) reading 
strategies for comprehension instruction 


Using Pairing for Partner Reading & Content Too 
(PRC2) for comprehension instruction 


How often do you use the following practices to help students build their vocabulary 
knowledge? 


1-3 times | 1-3 times 
amonth | a week 


% % 

Explicit instruction in vocabulary 131 8% 2.3% 5.3% 61.1% 30.5% 
Modeling the use of word parts 132 3.0% 8.3% 18.9% 48.5% 21.2% 
Review of vocabulary words 131 8% 1.5% 9.2% 59.5% 29.0% 
Use of vocabulary notebooks 129 15.5% 3.1% 19.4% 41.9% 20.2% 
PRC2 for vocabulary development 130 20.8% 15.4% 26.9% 29.2% 7.7% 
Use of before, during, and after (BDA) reading 130 9.2% 6.9% 20.0% 46.9% 16.9% 
strategies for vocabulary instruction 

Academic vocabulary for content terms (e.g., 132 3.8% 9.1% 21.2% 47.7% 18.2% 


Marzano) 


noice instruction (€.g., Shane 129 | 240% | 163% | 30.2% | 22.5% | 7.0% 
ue pinay sens ane) concepts (eiEs Donald 130 | 21.5% | 17.7% | 26.2% | 23.8% | 10.8% 
ae bay Way (eg. Donald Bear 4 Shane 130 30.0% | 14.6% | 21.5% | 20.0% | 13.8% 


Comprehensive Instruction 


In a typical classroom, how often do you use the following techniques to help students 
develop better reading strategies and skills? 


Never/ 
Not 1-3 times | 1-3 times | 4-5 times 
Familiar amonth | a week a week 


% % % % 


Everybody Reads Together (ERT) 
Exclusion Brainstorming 
List-Group-Label 
Predict-Locate-Add-Note (PLAN) 
ReQuest 

Interactive Notation System for Effective 
Reading and Thinking (INSERT) 

ABC Graffiti 


Guided Reading and Summarizing Procedure 
(GRASP) 
KWL 


Indicate how often (if at all) you discussed each of the following topics with any literacy 
experts during the current school year. 


1-3 4-5 
times a | timesa 
week week 


% 


Differentiated instruction 


Student groupings 

Use of Striving Readers text sets and text sets 
teacher guides 

Use of technology resources: desktop computers, 
handhelds-palms, LCD projector etc. 


Use of PRC2 instructional framework 


Using specific Striving Readers instructional 
techniques for comprehension instruction 


Using specific Striving Readers instructional 
techniques for vocabulary instruction 


Specific students' reading progress 
Using student assessment data for instructional 
planning 


School-wide Intervention Materials 


Do you use Striving Readers text sets to teach literacy through your content area? 


aa 


ee 


Of those not using Striving Readers text sets 


If you are NOT yet using Striving Readers text sets, please indicate why you are not using 
them below (Check all that apply). 


Not Checked | Checked 


They have not been made available to me 


The content is not relevant/interesting to my students 


The Striving Readers text sets are not sufficiently 
aligned to the curriculum of my subject area 


The range of reading levels covered by the Striving 


Readers texts sets is too high for my students 


The range of reading levels covered by the Striving 
Readers texts sets is too low for my students 


I did not receive Striving Readers texts sets for the 
topics that we are covering in my class 


I have Striving Readers texts sets, but not the text sets 
teacher guides 


Other 


Other (specify): (If you are NOT yet using Striving Readers text sets, please indicate why 
you are not using them.) 


Don't like 

I am a new teacher and due to my overwhelmed state this year, never organized myself to align my 
curriculum with the Striving Readers texts (though they seem amazing!) 

I believe they take away from instruction 


I don't have enough of them for even a small group- I have 32 kids. 


I use online reading resources 


Special education teachers do not receive text sets. They are available to me to borrow, and I have 
on a few occasions. 


These text are used by classroom teachers, I am a resource teacher. 


Time 


We use other materials in our resource program 
Total 


School-wide Intervention Materials 


Of those using Striving Readers text sets 


Please indicate the proportion of your students for whom the following statements are true. 


All or Hardly 
almost all All Most A few any 
students | students | students students | students 
% % % % % 


The range of reading levels covered by the 
Striving Readers text sets is too high for 
these students 

The range of reading levels covered by the 
Striving Readers text sets is too low for 
these students 

The range of reading levels covered by the 
Striving Readers text sets is appropriate 
for these students 

Striving Readers text sets are relevant to 
their interests 

Striving Readers text sets are appropriate 
to their learning style 

Striving Readers text sets motivate 
students to learn more about a topic 
Striving Readers text sets are appropriate 
to their literacy needs 


Number of Striving Readers Text Sets that Respondents Reported Using During SY 2009-10 


# of units being used 


School-wide Intervention Materials 


Subject Areas for which Striving Readers Text Sets Were Used: N and % of Text Sets 
Across All Respondents 


N and % of Text Sets 
Used for Each Subject 
Subject 
5. 
49. 


* Total exceeds the number of respondents because some respondents indicated using more than 
one text set applying to more than one instructional unit; in some cases these may have been 
used for more than one subject area. 


Please see report appendix for listing of text sets reported to be used in each subject area 


Rate how comfortable you are with using the Striving Readers text sets to support student 
learning in the language arts. 


1 Not at all 5 Very 
Comfortable Comfortable 


In your opinion, how well are the Striving Readers text sets aligned to the curriculum of 
your subject area? 


3 5 Very 
1 Poorly Adequately well 
Aligned Aligned aligned 


| od NN % % % % % 
PJs 9% 59% | 8a | 255% | 973% 
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Professional Development in Literacy Instructional Practice 


Did you receive professional development (PD) addressing this topic? 


Building academic vocabulary 


Using classroom libraries 


Creating literacy-rich classroom environments 


Differentiating instruction 


Explicit vocabulary instruction 
Incorporating text sets and teacher guides in your 
instruction 


Increasing student motivation 


Supporting students' self-directed learning 


Jsing before, during, and after reading strategies 


sing formal assessments to guide instruction 


sing informal assessments to guide instruction 


sing handheld computers (Palm Pilots) 


sing literacy-based software 
sing the PRC2 model 
sing the whole-part-whole classroom instruction model 


Building academic vocabulary 


Using classroom libraries 


Creating literacy-rich classroom environments 


Differentiating instruction 


Explicit vocabulary instruction 

Incorporating text sets and teacher guides in your 
instruction 

Increasing student motivation 


Supporting students' self-directed learning 


Jsing before, during, and after reading strategies 


sing formal assessments to guide instruction 


sing informal assessments to guide instruction 
sing handheld computers (Palm Pilots) 


sing literacy-based software 
sing the PRC2 model 
sing the whole-part-whole classroom instruction model 


Professional Development in Literacy Instructional Practice 


If you received and are using this PD as part of content-area instruction, rate your comfort 


implementing each teaching practice. 


1 -- Not at 
all 
Comfortable 


5 -- Very 
Comfortable 


Building academic vocabulary 


% 


% 


Using classroom libraries 


Creating literacy-rich classroom 
environments 


Differentiating instruction 


Explicit vocabulary instruction 


Incorporating text sets and teacher 
guides in your instruction 


Increasing student motivation 


Supporting students' self-directed 
learning 


Using before, during, and after 
reading strategies 


Using formal assessments to guide 
instruction 


Using informal assessments to 
guide instruction 


G 


sing handheld computers (Palm 
Pilots) 


sing literacy-based software 


U 

Using the PRC2 model 
Using the whole-part-whole 
classroom instruction model 
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Professional Development in Literacy Instructional Practice 


Please check the techniques in the list below for which you would like to receive more 
training. (Check all that apply.) 


Academic Vocabulary for content terms (e.g., 
Marzano) 
Morphology instruction (e.g., Shane Templeton) 


Word study sorts and concepts (e.g., Donald Bear) 
Words Their Way 

KWL 

Using PRC2 for fluency instruction. 


Using PRC2 for comprehension instruction. 


Using PRC2 for vocabulary development. 
Everybody Reads To (ERT) 

Exclusion Brainstorming 
List-Group-Label 


Predict-Locate-Add-Note (PLAN) 


ReQuest 


Interactive Notation System for Effective Reading and 
Thinking (INSERT) 


Read Aloud/Think Aloud 
ABC Graffiti 


Guided Reading and Summarizing Procedure 
(GRASP) 


Teaching summarizing as a comprehension strategy 


Teaching questioning as a comprehension strategy 


Teaching predicting as a comprehension strategy 


Teaching text structure as a comprehension strategy 


Teaching visualization as a comprehension strategy 


Teaching inferring as a comprehension strategy 


Teaching metacognition as a comprehension strategy 


Respondent Information 
What is your position? 


General 

Bilingual/ Special Education/ 
General ELL Education Reading Bilingual/ELL 

Education | teacher Teacher Specialist teacher 


% % % % % 


Computer Teacher 


Departmental Language Arts/SS teacher 


Science Teacher 


Technology 
Total 


What subjects do you teach? 


Not Checked Checked 
N 


All subjects 
Literacy/Reading/English language arts 


Mathematics 


Science 


Social studies 


Other (Please specify) 


Advisory, Spanish Language Arts 


Computer 


Computer Technology 


Special Ed. Upper Grades 


Technology 


Writing 
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Respondent Information 


At which grade level(s) are you teaching English language arts this year? 


Not 
Checked 


Checked 


% 


% 


OPW! NADI MHWL NY HLA 


10 
11 


— 
No 


How many years have you been teaching? 


How many years have you been teaching at this school? 


How many years have you been teaching math?* 


How many years have you been teaching science?* 


How many years have you been teaching social studies?* 


How many years have you been teaching English language 


arts?* 


*Includes respondents who indicated that they teach all subjects. 
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Appendix: Text Sets 


Text Sets used during instruction 


Text Sets used for Instructional Units in Mathematics (N=8) 


* These respondents indicated using this text set for units in a subject area other than the one for 
which it may have been primarily intended. 


Text Sets used for Instructional Units in Science (N=61) 
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Appendix: Text Sets 


Text Sets used for Instructional Units in Social Studies (N=74) 


Ss 


Text Sets Used 
Not Described 
American Revolution 


i) 


36 48.6% 
4.1% 
14% 
2.7% 
8.1% 
14% 
14% 
1.4% 
6.8% 
1.4% 
2.7% 
1.4% 
1.4% 
1.4% 
14% 
1.4% 
1.4% 
14% 
2.7% 
1.4% 


Ancient civilizations 


N 


Ancient Egypt 


Ancient Greece 


| Ancientcivilizations 


Beginning America 
Civilizations Past to Present China 
Civilizations Past to Present Rome 


— 


nN 


Constitution 


—_ 


Earthquakes* 

Egypt 

Egypt, Nubia and Kush 
Finding the Pharaoh 
Greece 


re Th 


Habitats, ecosystems* 
India in the Past and Present 
Industrial Revolution 


— 


Land and Resources of Ancient Greece 
National Geographic Reading Expectations, Graphic Discoveries 


Obama/ Beyonce/ 50 cents 

Rosen Classroom Primary Source, National Geographic Reading Expeditions, Graphic 
Mythologies 

The Animals* 

The Bibliography/Nonfictions 

The Constitution 1.4% 
Vietnam, Korea, WW2 1.4% 

* These respondents indicated using this text set for units in a subject area other than the one for 
which it may have been primarily intended. 


1.4% 
14% 
1.4% 


— 


Text Sets used for Instructional Units in Other Content Areas (N=7) 
Text Sets Used 
Not Described 
Boldprint 
Books on data 
Ecology* 


14 
It's a Goal 14.3% 
14 


scholastic 
The Fictional/ Nonfiction Texts 
* These respondents indicated using this text set for units in a subject area other than the one for 
which it may have been primarily intended. 
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CPS Striving Readers Evaluation: 
Spring 2010 Literacy Improvement Survey Results 


All Control Schools: Content Area Teachers* 


* Content Area Teachers are defined as those teachers who provide instruction in mathematics, science 
and/or social studies. These may include those who teach literacy or common branch teachers, as well as 
special education and/or ELL teachers. 


S' metis associates 


90 Broad Street 
Suite 1200 
New York, NY 10004 
212-425-8833 
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Survey Response Rates: 


school | Response | aetoondents | “Ratt 
Overall 70 159 44% 
Aldridge 3 3 100% 
Carnegie 0 9 0% 
Carver* 0 9 0% 
Casals 3 4 75% 
Clark 2 3 67% 
Dubois 1 3 33% 
Copernicus** 0 7 0% 
Dvorak 0 7 0% 
Emmet al 5 20% 
Esmond 1 5 20% 
Gregory 0 3 0% 
Henderson 1 5 20% 
Madison 3 5 60% 
Mann 1 6 17% 
McKinley 1 3 33% 
Mireles 7 10 70% 
Morgan a 3 33% 
O'Keefe 2 5 40% 
Otis 4 6 67% 
Parkman 0 3 0% 
Pasteur 8 12 67% 
Pullman 4 2 200% 
Reinberg 6 9 67% 
Schiller*** be zs 7 
Sexton 1 3 33% 
South Chicago*** -- 7 a 
Spry 4 7 57% 
Swift 7 6 117% 
Turner-Drew 4 3 133% 
Wacker 0 3 0% 
Walsh 0 5 0% 
Whistler 5 5 100% 


t Response rates exceed 100% at some schools where surveys were received from more content area teachers than 


were indicated in the district's records. 


* Carver Middle School stopped participating after it consolidated with Carver Elementary School as of SY09. In 


order to reflect the original random selection of participating schools, however, it is still included in the calculation of 


response rates. 


*** No responses were received from Copernicus because it had been identified as a turnaround school as of the 
2009-2010 School Year. In order to reflect the original random selection of participating schools, however, it is still 


included in the calculation of response rates. 


*** Schiller and South Chicago, two of the original Striving Readers schools, closed after SY09. 


When, if at all, did you start integrating literacy into your 
content area instruction? 


I do not 
integrate 
literacy 
Two or instruction 
Last school | more years into my 
ral ee seal ear ear ago content a.. 


ee 55.7% 


Metis Associates 
All Control Schools: Content Area Teachers 
Page 1 
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Comprehensive Instruction 


In a typical classroom, how often do you use the following teaching practices to help students increase reading comprehension? 


Less than 
Not oncea 1-3 times a 
Familiar month week 


% % % 


Explicit instruction in the use of any one or more of the following 
comprehension strategies: summarizing, questioning, predicting, text 
structure visualization, inferring and metacognition 


Establishing the purpose of reading 


Monitoring students' comprehension through questioning 


Making connections to background knowledge 


Making connections between texts 


Synthesizing information within text or across texts 64 0% 4.7% 1.6% 
Using differentiated instruction 63 0% 1.6% 1.6% 
Use of before, during, and after (bBDA) reading strategies for 
comprehension instruction... 


Using partner reading to enhance comprehension instruction 


How often do you use the following practices to help students build their vocabulary knowledge? 


Less than 
Not once a 1-3 times a | 1-3 timesa 
Familiar month month week 


% % % % 


Explicit instruction in vocabulary 63 
Modeling the use of word parts 62 


Review of vocabulary words 


Use of vocabulary notebooks 


Use of partner reading to enhance vocabulary development 


Use of before, during, after (BDA) reading strategies for 
vocabulary instruction 


Academic vocabulary for content terms (e.g., Marzano) 


Morphology instruction (e.g., Shane Templeton) 


In a typical classroom, how often do you use the following techniques to help students develop better reading strategies 
and skills? 


Less than 
Not oncea 1-3 times a 4-5 times a 
Familiar month month week 


% % % % 


Everybody Reads Together (ERT) 
Exclusion Brainstorming 
List-Group-Label 
Predict-Locate-Add-Note (PLAN) 
ReQuest 


Interactive Notation System for Effective Reading and 
Thinking (INSERT) 


ABC graffiti 
Guided Reading and Summarizing Procedure (GRASP) 
KWL 


Metis Associates 
All Control Schools: Content Area Teachers 
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Comprehensive Instruction 


Indicate how often (if at all) you discussed each of the following topics with any literacy experts during the current 
school year. 


Less than 
once a 1-3 times a -3 ti 4-5 times a 
Total Never month month week 
N % % % % 


[Differeniatedinstnicion ————S—S—SC~CSSCSSCCTCCS 8 —*«Y~Si%—| am | 27.5% | 29.0% | 17.4% 
Student groupings | 68 | 10.3% 16.2% 23.5% 27.9% 22.1% 
68 ; 


se of text sets 


Use of technology resources: desktop computers, classroom 
computers/laptops, LCD projector, etc. 


Use of partner reading instructional technique 


Using specific instructional techniques for comprehension 
instruction 


Using specific Instructional techniques for vocabulary instruction 22.1% 11.8% 29.4% 27.9% 8.8% 
Specific students' reading progress 22.1% 11.8% 32.4% 25.0% 8.8% 


Using student assessment data for instructional planning a 16.2% 35.3% 27.9% 14.7% 


Metis Associates 
All Control Schools: Content Area Teachers 
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School-wide Intervention Materials 


Do you use Striving Readers text sets to teach literacy through your content area? 


% 


A 


Of those not using Striving Readers text sets 


If you are NOT yet using Striving Readers text sets, please indicate why you are not 
using them below (Check all that apply). 


They have not been made available to me 


Other: If you are not using 
text sets, please indicate 
why you are not using 
them below 


Of those using Striving Readers text sets 


Please indicate the proportion of your students for whom the following statements are true. 


All or 
almost all Most A few Hardly any 
Total students All students students About half students students 


N 
The range of reading levels covered by text sets is too high 29 .0% 


The range of reading levels covered by text sets is too low 29 3.4% 


The range of reading levels covered by text sets is appropriate 28 7.1% 
Available text sets are relevant to their interests 


Available text sets are appropriate to their learning style 


Available text sets motivate students to learn more about topic 


Available text sets are appropriate to their literacy needs 


Metis Associates 
All Control Schools: Content Area Teachers 


Page 4 L-21 


School-wide Intervention Materials 


Number of Text Sets that Respondents 
Reported Using During SY 2009-10 


# of units being used 


Respondents |Respondents 


Not Using/None Identified 74.3% 
a COSSSCSCS~SYCi‘“‘C”#SCCMW 


a aE EZ 
ee 


Subject Areas for which Text Sets Were 
Used: N and % of Text Sets Across All Respondents 


N and % of Text Sets Used for 
Subject Each Subject 


Man fo ———ioow —_—t 
aaa —S—id aid 


* Some respondents indicated using more than 
one text set applying to more than one 
instructional unit; in some cases these may have 
been used for more than one subject area. 


**These respondents did not indicate specific 
subject areas. 


Rate how comfortable you are with using the available text sets to support 
student learning in the language arts. 


ag Fe ea 
Total Comfortable 2 3 Comfortable 


0 0 0 
P| om «Tae «| r2% | 17.2% | 62.1% 


Metis Associates 
All Control Schools: Content Area Teachers 
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School-wide Intervention Materials 


In your opinion, how well are the available text sets aligned to the curriculum of your subject area? 


Poorly Adequately Very well 
Total Aligned 2 Aligned aligned 


Pe (10% 31.0% | 51.7% 


Metis Associates 
All Control Schools: Content Area Teachers 
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Professional Development in Literacy Instructional Practices 


Did you receive professional development (PD) addressing this topic? 


| Total | Yes | No 
Se ee Se 
Building academic vocabulary 
Using classroom libraries 
Literacy-rich classroom environments 
Differentiating instruction ‘ 
Explicit vocabulary instruction 
Incorporating text sets 
Student motivation 
Students' self-directed learning 
50 ; 
50 0 , 


Before, during, after reading strategy 
Formal assessments to guide instruction 
Informal assessments to guide instruction 
Literacy-based software 
Partner reading | 50 | 46.0% | 54.0% 


Whole-part-whole model | 50 || 40.0% 60.0% 


Ni 
Lt) 
39 60 
84.9% 15.1% 
4 5 
13 
4 5 


Using classroom computers 


Yes 
% 


Are you using the literacy based teaching practice as part of content-area instruction? 
4 
4 
4 
4 


es N 
% 0, 
92.3% 


Building academic vocabulary 
27) 


Using classroom libraries 


84.2% 15.8% 


Literacy-rich classroom environments 
94.0% 


Differentiating instruction 


Explicit vocabulary instruction 


| 
7 
19 
Incorporating text sets 
Student motivation 
Literacy-based software 
Partner reading 
Whole-part-whole model 


9 
Students' self-directed learning 
Before, during, after reading strategy 
Formal assessments to guide instruction 9 
Informal assessments to guide instruction 


Using classroom computers 


72 
0 
3 
0 


Metis Associates 
All Control Schools: Content Area Teachers 
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Professional Development in Literacy Instructional Practices 


If you received and are using this PD as part of content-area instruction, rate your comfort implementing each teaching practice. 


Total comfortable 2 3 4 comfortable 

Building academic vocabulary 
Received PD and Are Using The Teaching Practice | sing classroom libraries 14 0% 7.1% 35.7% 21.4% 35.7% 
Received PD and Are Using The Teaching Practice | Literacy-rich classroom environments 26 0% 3.8% 30.8% 26.9% 38.5% 


Received PD and Are Using The Teaching Practice | Differentiating instruction 0% 7.3% 26.8% 19.5% 46.3% 
Received PD and Are Using The Teaching Practice } Explicity vocabulary instruction 


41 
i 21 4.8% 4.8% 19.0% 28.6% 42.9% 
Received PD and Are Using The Teaching Practice | Incorporating text sets 13 0% 0% 23.1% 30.8% 46.2% 
Received PD and Are Using The Teaching Practice | Student motivation 27 .0% .0% 25.9% 29.6% 44.4% 
21 


Received PD and Are Using The Teaching Practice | Students' self-directed learning 0% 4.8% 33.3% 23.8% 38.1% 
Received PD and Are Using The Teaching Practice | Before, during, after reading strategy 24 0% .0% 45.8% 25.0% 29.2% 


Received PD and Are Using The Teaching Practice | Formal assessments to guide instruction 


Received PD and Are Using The Teaching Practice | Informal assessments to guide instruction 


Received PD and Are Using The Teaching Practice | Using classroom computers 
Received PD and Are Using The Teaching Practice | Literacy-based software 
Received PD and Are Using The Teaching Practice } Partner reading 

Received PD and Are Using The Teaching Practice | Whole-part-whole model 


Metis Associates 
All Control Schools: Content Area Teachers 
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Respondent Information 


What is your position? 


General Education Teacher 


Bilingual/ELL Teahcer 


Not 
Total Checked Checked 


3 
f 


Other - What is your position? 


Count 
3 
Other Middle School Science Teacher 
(Pleas 
Specify) | Science teacher 
Science Teacher 
Writing teacher 


What subjects do you teach? (Check all that apply) 


Not Checked Checked 


Asc CR 
foie SSCS Cd CC CP CCC” 


Other - What subjects do you teach? 


PO ournt _ 
Other: (Please specify) (What subjects(s) | Spanish 6-8 


do you teach? (Check all that apply)) 


Metis Associates 
All Control Schools: Content Area Teachers 
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Respondent Information 


At which grade level(s) are you teaching English language arts this year? 


See 
Total Checked Checked 


PN 
0 | oe 
[70 | aa. 

P70 [98.6% 

P70 [98.6% 

P70 [95.7% 

P70 [92.9% 

P70 [sia | 08.6% | 
P70 [29% [57.1% | 
P70 [47% | 52.9% | 
P70 [100.0% [0% | 
P70 [100.0% [0% | 
F709 [100.0% [0% | 
P70 [100.0% [0% | 


Experience 


a 
Flow many years have you been teaching? a 


How many years have you been teaching at this school? aa ee CS eT 
How many years have you been teaching in your subject area? | 6 =| 1 [| 36 | 95 | 


Metis Associates 
All Control Schools: Content Area Teachers 
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CPS Striving Readers Evaluation: 
Spring 2010 Literacy Improvement Survey Results 


All Treatment Schools: Literacy Teachers* 


* Respondents may include general education teachers, English language arts teachers, bilingual/ELL 
teachers, special education teachers, reading specialists and/or other teachers who indicated that their 
role included the teaching of English language arts in addition to other subject areas. 


metis associates 


90 Broad Street 
Suite 1200 
New York, NY 10004 
212-425-8833 


L-28 


Survey Response Rates: 


School over ites Response Ratet 
Overall 167 121 72.5% 
Abbott* E P NA 
Beethoven ES 5 3 60.0% 
Bethune** 4 : 0.0% 
Burr 5 4 80.0% 
Burroughs 6 2 33.3% 
Carson 11 7 63.6% 
Colemon 3 4 133.3% 
Coles 6 7 116.7% 
Cook 4 7 175.0% 
Dett 3 3 100.0% 
Eberhart 18 8 44.4% 
Fiske 4 4 100.0% 
Fuller 2 2 100.0% 
Gale 7 2 28.6% 
Gompers 6 6 100.0% 
Gray 11 5 45.5% 
Hendricks 3 5 166.7% 
Henson 5 6 120.0% 
Linne 6 5 83.3% 
Lovett 6 2 33.3% 
Manierre 8 5 62.5% 
Marsh 8 8 100.0% 
McCorkle 2 2 100.0% 
Pope 3 3 100.0% 
Price 1 0 0.0% 
Reavis 2 3 150.0% 
Salazar 4 4 100.0% 
Smyth 9 0 0.0% 
Talcott 3 2 66.7% 
Telpochcalli 4 3 75.0% 
Volta 8 9 112.5% 


qResponse rates exceed 100% at some schools where surveys were received from more literacy teachers than were 


indicated in the district's records. 


* Abbott, one of the original Striving Readers schools, was not included in this survey administration because it closed 


after SYO9. 


**No responses were received from Bethune because it had been identified as a turnaround school as of the 2009- 
2010 School Year. In order to reflect the original random selection of participating schools, however, it is still 


included in the calculation of response rates. 
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Comprehensive Instruction 


How often do you use the following teaching practices to help students increase reading comprehension? 


Less than 
oncea 1-3 timesa | 1-3timesa | 4-5 timesa 
month month week week 


N % % % % % 
Explicit instruction in the use of...summarizing, questioning, a 
predicting, text structure visualization, inferring and 119 0% 0% 1.7% 23.5% 74.8% 
metacognition 
Establishing the purpose for reading 
Monitoring students' comprehension through questioning 


Making connections to background knowledge 119 .0% 32.8% 63.9% 
Making connections between texts 120 .0% 11.7% 50.0% 38.3% 


Synthesizing information within text or across texts 120 


Using differentiated instruction 119 


Using before, during, after (BDA) reading strategies for 1 
comprehension instruction 


20 : 
Using Pairing for Partner Reading & Content Too (PRC2) for 121 5.0% 26.4% 42.1% 18.2% 
comprehension instruction 


How often do you use the following practices to help students build their vocabulary knowledge? 


Less than 
oncea 1-3 timesa | 1-3 timesa 
Total Never month month week 


Use of before, during, and after (BDA) reading strategies 
for vocabulary instruction 


How often do you use the following practices to help students develop fluency? 


Less than 
oncea 1-3 timesa | 1-3timesa | 4-5 timesa 
Total Never month month week week 
N % % y 


% % % 
Teacher read aloud 1.7% 9.9% 50.4% 38.0% 


Teacher interactive read aloud 120 


Shared reading (students and teacher take turns in reading) 120 


Modeling reading for students 119 


Explicit instruction in guided oral reading 4.3% 16.2% 41.9% 36.8% 
Students listen to audio books, play aways 121 12.4% 11.6% 28.1% 41.3% 


Metis Associates 2010 
All Treatment Schools: Literacy Teachers 
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Comprehensive Instruction 


How often do you use the following techniques to help students develop better reading strategies and skills? 


Less than 
Never/ Not once a 1-3 timesa | 1-3 timesa 
Familiar month month week 


% % % 
Evens Reads To ee) 36.8% 11.1% 17.1% 24.8% 10.3% 


ReQuest 


Interactive Notation System for Effective Reading and 
Thinking (INSERT) 


ABC Graffiti 


Less than 
once a 1-3 timesa | 1-3timesa | 4-5timesa Multiple 
month week week times a da 


% % % % 
who dasiageGeup | 10 | 17% | a6 | 25% | 25.0% | 325% | 37.5% 
Findvidval Work [119 | 0% | 0% | 25% | 244% | 33.6% | 395% | 


Considering your own instruction (not that of the LIT or other instructors in your classroom), 
in a typical classroom, how often do you apply differentiated instruction in your classroom? 


Almost 
About half Most of the | every lesson 
a ae ee =F time 


a 


Metis Associates 2010 
All Treatment Schools: Literacy Teachers 
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Purposeful Assessment 


Indicate whether you use the data from the following assessments. (Please check all that apply.) 


Valid N 


Reading Benchmark Assessment 94.2% 5.8% 


Spelling Inventories 


1 
1 
1 
1 
1 
1 
1 


Other types of assessment data. 


ACCESS Language Proficiency for ELLs 


3 


| 
i 
eee 
| 


www.bookadventure.com 


Metis Associates 2010 
All Treatment Schools: Literacy Teachers 
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Purposeful Assessment 


Of those who are using each assessment... 


Indicate how you use Reading Benchmark Assessments 


Not using for this purpose 
Screening Foo | 76.1% | 25 | 21.9% | 
11d 


Not using for this purpose 
a 
Psa i| 83% | 20 ~—~|~—*16.9% | 
| 

2 41% | 66 | 55. 


3 
55.1% 


Screening 65.1% 
79 2 


Not using for this purpose 
45. 
Assessing Outcomes [79 725% 


Screening | 686 72.3% 


Not using for this purpose 


Metis Associates 2010 
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Purposeful Assessment 


Of those who are using each assessment... 


Indicate how you use Fluency Snapshots 


Not using for this purpose 
Screening 
119 


Screning 64.1% 
P90 | 760% | 27 | 23.1% 


Not using for this purpose 
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Data Driven Instruction 


Indicate the extent to which you use student assessment data for each of the following purposes 


Toa 
To Some Moderate | Toa Large 
Total Not at All Extent Extent Extent 
Valid N % 


Placing students in intervention programs 7.6% 24.4% 
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Grade Level Teams 


Do you currently have grade-level teams at your school? 


Pt 97.5% 


Overall, rate the grade-level team’s ability to use classroom assessment data in the following ways. 


Total __/_Poor _{_Fair Good Excellent Not Sure 

a 
Address the literacy needs of all students 39.0% 50.0% 8% 
Address the needs of struggling readers | 118 | 0%) 7.6% 43.2% 48.3% 8% 
Formalize lesson plans 43.2% 33.1% 2.5% 
Identify students who eligible for targeted interventions 118 0% 8.5% 36.4% 53.4% 1.7% 
35.3% | 543% | 9% 
37.9% | 50.9% | 17% 
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Literacy Teams 


Do you currently have vertical literacy teams at your school? 


Pt 98.7% 
Overall, rate the quality of the literacy team’s performance in the following areas. 


ee ses 


| ValidN | N 


Using assessment data to pinpoint the staff's 2.6% 12.0% 35.0% 48.7% 1.7% 
piviessional development needs 


ee 
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School-wide Intervention Materials 


Indicate how frequently you currently use the following materials to teach literacy. 


Less than 
N/A (Do rate once a 1to3 times | 1to3 times | 4 to5 times 
Not a ae ey Canes =a a SES a week 


Media Centers 5.8% 2.5% 8% 17.5% 36.7% 36.7% 
Ses notebooks 9.2% 9.2% 1.7% 13.3% 49.2% 17.5% 
School library 8.4% 6.7% 7.6% 21.8% 43.7% 11.8% 
Reading Anthologies 20.5% 20.5% 25.6% 6.8% 
Reading Basals 9.4% 
Other informational texts (other than text sets) 9% 


Not at all Very 
Comfortable Comfortable 


% 


Media Centers 


Classroom library 


eearulay notebooks 


isi esig topeee aga a8 
Using [School brary ————SSSSCSC~C~S~C 
Using [Reading Bassls ——S~C~S~iC id 
[Using [Other information exis otherthanextses) [60 _| 
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School-wide Intervention Materials 


Of those who are using each material... 


Indicate how you use Listening Centers 
Not using for this purpose 
39.8% 
Reading Comprehension : 
87.4% 
i 


Vocabulary 63.1% 
Writing Skills 


67.0% 
To develop students’ self-directed learning 


-di i 50.5% 
To supplement students’ textbook reading 


5 
1 
0 
3 90.3% 
6 
1 
2 


P96 |_ 93.2% 


Teaching students to identify and use text structure 76.7% 
Teaching students to identify and use the 81.6% 
organizational features of expository writing . 


To activate students’ prior knowledge 72 69.9% 


6 
4 
3 
9 
9 
5 
4 
8 


N 
7 
79 

4 


9 

3 
88.396 

8 

1 


Vocabulary 


59.4% 


44.9% 


62 
75.4% 
75 

0 


Teaching students to identify and use text structure 
Teaching students to identify and use the 
organizational features of expository writing 


To activate students’ prior knowledge 


Reading Comprehension j 


a 
| 69 =| 
ee 
| 69 =| 2 
| 69 =| 2 
| 69 =| 
[Spelling 88.7% _| 
| 69 =| 82 
| 69 6 | 39 
a ee 
ee ee 
| 69 =| 
ae | 
a 
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School-wide Intervention Materials 


Of those who are using each material... 


Indicate how you use the Classroom Library 
Not using for this purpose 
38.39% 
Reading Comprehension : 
70.2% 
7 


Vocabulary 40.4% 
Writing Skills 
76.6% 


Word Recognition 
To teach content themes 


To develop students’ self-directed learning 
To supplement students’ textbook reading 


7 

1 
44.7% 
29.8% 
31.9% 
Teaching students to identify and use text structure 59.6% 
Teaching students to identify and use the OLY 
organizational features of expository writing . 

6 


To activate students’ prior knowledge 22 


1 
72.3% 
70 


aed 
a 


Vocabulary 37.5% 


p 

| 
Teaching students to identify and use the 

; 


To activate students’ prior knowledge 


72.7% 


53.4% 
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School-wide Intervention Materials 


Of those who are using each material... 


Indicate how you use the Reading Anthologies 
P60 | 08.8% _| 
p69 | 80.2% 
: 
organizational features of expository writing : 
| 608. 


To activate students’ prior knowledge 


5 
69.8% 


Vocabulary 


63.2% 


54.7% 


Oo, 
6 
68.4% 
4 
7 


Teaching students to identify and use text structure 
Teaching students to identify and use the 
organizational features of expository writing 


To activate students’ prior knowledge 


46. 
Reading Comprehension F 


re ee 
ps | wo 


Metis Associates 2010 
All Treatment Schools: Literacy Teachers 
Page 12 L-41 


School-wide Intervention Materials 


Of those who are using each material... 


Indicate how you use the School Library 


Not using for this purpose 


60.0% 
48.2% 


Vocabulary 


Reading Comprehension 


| 
Writing Skills 


i 
85 
85 


To supplement students’ textbook reading 


Teaching students to identify and use text structure 
Teaching students to identify and use the 
organizational features of expository writing 


To activate students’ prior knowledge | 85 


32 

| 

| 60 || 70.6% 
| 68 | 80.0% 
8 

" 

8 


Indicate whether you use each of these materials in your classroom to 
support grouping strategies or differentiated instruction 


19.0% 


95.9% 5 


19 
16.5% 
8 


N 
2 
1 

4 
2 
2 
2 
4 
3 


4 
1 
3 
0 
4 
6 
3 


| 8 | 66% 
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School-wide Intervention Materials 


Of those who are using each material... 


Indicate how you use Listening Centers to support grouping strategies or differentiation 


[Whole Chas Large Group | 97 
Differentiated nsinction | 97 


110 
110 
110 


17 
17 
117 


Whole Class / Large Group 


Small Group / Pairs 
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Not using for this purpose 


76.8% 
P68 | 58.1% 
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School-wide Intervention Materials 


Of those who are using each material... 


Indicate how you use Vocabulary Notebooks to support grouping strategies or differentiation 


oo = a 7 = cane 
%o 


Not using for this purpose 
113 
113 
113 


Small Group / Pairs 
Individual Work 
Differentiated Instruction 


Whole Class / Large Group 
97 


Differentiated Instruction | 97 


1 
38.1% 
43.3% 


Not using for this purpose 
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School-wide Intervention Materials 


Of those who are using each material... 


Indicate how you use Reading Anthologies to support grouping strategies or differentiation 


= 
Whole Class / Large Group 


Small Group / Pairs 29. ae 
Individual Work 37. = 
Differentiated Instruction 


Indicate how you use Other informational texts (other than text sets) to 
support grouping strategies or differentiation 


= “Ne a 


Whole Class / Large Group aE 


Small Group / Pairs 31. = 
Individual Work 
Differentiated Instruction 29, oe 
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Handheld Computers 


Do you use handheld computers (Palm Pilots) to teach literacy? 


Tar save a8. 


Reasons for not using handhelds 


I have not received the handheld computers — 0 ae 


Some or all of the computers are not working properly —— 89.1% 10.9% 
Some or all of the necessary software applications have 94.5% 5.5% 
not been installed on the computers 

I have not received sufficient professional development to 63.6% 6.4% 
feel comfortable using them 


Because the Striving Readers program provides only 10 
94.5% 5.5% 


computers per classroom, and I do not like to have some 
students use them while others can not 


I do not feel that they offer sufficient added benefit 
compared to traditional media (e.g., print, paper and 55 74.5% 25.5% 
pencil) to be worth the trouble 
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Handheld Computers 


Other reasons for not using handhelds. 


x 


All 7th grade students have their own laptop 

Bilingual students need more time, and we have them for 
limited time with them 

Handhelds are locked in a school closet 

Have been out on leave for most of the year, did not have time 
when returned 


Other :If 
I'm not sure 


you are 
NOT yet I don't feel they're worth the trouble. Too toy-ish and I'd rather 
using use the computer or traditional graphic organizers... 


handheld I have only one 


computers, 


please 
indicate Lack time during class period 


I teach science only 


why you My students use them in their inclusion classrooms in the 
are not content areas 


using them. | Not available all the time 


Problems working 


The computing power of our classroom desktops and laptops is 
much more efficient and requires a lot less time to set up 


They were used in Social Studies 


We share a set between three grade levels 


Of those who are using handheld computers... 


Which grouping strategies do you support through the use of 
handheld computers (Palm Pilots) during literacy lessons? 


Not using Using for 
for this this 


grouping 


% 
Whole class/ Large group | 66 27.3% 72.7% 
Small group/ Pairs | 666 10.6% 89.4% 
Individual Work | 66 | 24.2% 75.8% 


In a typical classroom, how often do students use handheld computers 
(Palm Pilots) during literacy instruction in your classroom? 


Less than 
oncea 1-3 timesa | 1-3timesa | 4-5 timesa 
month week week 
% 


N % 


Ps | 6.5% | 50.800 
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Handheld Computers 


Rate how comfortable you are with using the Palm Pilots to support your literacy instruction. 


Total Comfortable 2 3 Comfortable 


PT sm | 5.6% | 42.2% | 17.2% | 219% 


Specific academic foci or instructional objectives supported by the use of handheld computers 


[reel ered eee 
Total Checked Using 


Fluency P66 | 621% | 37.9% | 
Vocabulary development | 666 18.2% =| 81.8% 
Developing students' reading comprehension strategies | 666] 57.6% =| 42.4% 
Writing skills | 666 39.4% 60.6% 
| 66 | 37.9% | 62.1% 
| 6s | 56.1% | 43.9% 
| 66 | 60.6% | 39.4% 
| 66 | 66.7% | 33.3% 
Locating information | 666 40.9% 59.1% 
Evaluating information | 66 | 65.2% 34.8% 

a 

| 6s | 3A. 

ee 

L 166- i] 37 

| ss. 


Word parts 

Word recognition 
Spelling 
Grammar 


Organizing information 


4 
34.8% 65.2% 
53.0% 47.0% 

62.1% 


45.5% 54.5% 


Demonstrate knowledge of key concepts 


To develop students’ self-directed learning 


Teaching students to identify and use the organizational 
features of expository writing 


6 
66 
Synthesizing information 45.5% 54.5% 
7 
68 


To activate students' prior knowledge 


Specific instructional methods supported by the use of handheld computers 


ee 
Total Checked Checked 
Monitoring distribution and completion of assignments 77.3% 22.7% 
78.8% 21.2% 
71.2% 28.8% 
36.4% 63.6% 
53. 7. 
0. 50. 


Assessing students' literacy skills 
Monitoring students' progress 


Differentiating instruction for struggling readers (Tiers 2 & 3) 


Guided reading 


Partner reading 
Individual reading 


Differentiating instruction for English language learners/special 
education students 


Book club discussions 
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Handheld Computers 


How frequently do students in your typical classroom currently use the following handheld programs during 
literacy instuction? 


I : = Ee 
N/A (Do Currently once a 1 to 3 times | 1 to 3 times 
Not Have Using month a month a week 


9 0 
iKWL | 66d . 5.8% 33.3% 
Freewrite 62 ; 25.8% 21.0% 27.4% 4.8% 
, | 
62 
58 
8 


MS Word : 21.0% 27.4% 27.4% 11.3% 

MS Excel —2— 43.1% 15.5% 20.7% 

| 

51.7% 

12.9% 
PAAM management software application | 60 | 20.0% | 60.0% | 100% [| 50% | 33% | 1.7% | 
19.0% | 50.0% 


If using, rate how comfortable you are with using each software application. 


Total Comfortable 2 3 4 Comfortable 


33.3% | 12.1% | 39.4% 

Freewrite 27 7.4% 7.4% 33.3% 22.2% 29.6% 

PiCo Maps 25 8.0% 20.0% 28.0% 4.0% 40.0% 
26.3% | 21.1% | 15.8% | 10.5% | 26.3% 
12.9% | 32.3% | 9.7% | 35.5% 
sig 31.4% | 20.0% | 37.1% 
sng 103% | 41.4% | 10.3% | 31.0% 
sing 143% | 19.0% | 28.6% | 143% | 23.8% 
18.2% | 18.2% | 22.7% | 9.1% | 31.8% 
28.1% | 15.6% | 43.896 
10.7% | 32.1% | 143% | 35.7% 
PAAM management software application 47.1% 17.6% 5.9% 11.8% 17.6% 
31.0% | 273% | 9.1% | 136% | 182% 


el ere 


sing 
sing 


cia 


U 


rTEIAELF 


ec 
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Classroom Library 


Please check the ways that you use your classroom libraries. (Check all that apply.) 


Not 
Total Checked Checked 


: 
For content area instruction 


How do you use Interest Inventories? 
Do you use interest inventories to help 91.7% 
students self select reading material? in 


Do you use interest inventories to guide your io 
: 92.6% F 
purchases for the classroom library? 


Please indicate how true each of the following statements are about the organization of 
books in your classroom library 


...4s easily accessible to students 
...4s well organized and in good shape 
...has a checkout system in place 


...includes a variety of reading materials that are 
appropriate for readers of differing abilities 


...includes a variety of reading materials that 
appeal to readers with differing interests 
...has reading materials grouped by genre 


...has reading materials clearly labeled 
...has both nonfiction and fiction books 119 92.4% 


To what extent are you able to consider students when ordering books and other reading 
material with Striving Readers funds for your classroom library? 


Toa 
To a small moderate To a large 
extent extent extent Don’t know 
% % % 


i eae ae ae 
Sudents’ needs and reading abilities 121 8% 87.6% 
38.1% 
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School Library 


In a typical classroom, how often do you take your class to the library? 


Sometimes 
Rarely (less (at least Often (at Almost 
than once a least once a 


% 
4.2% 


To what extent do the library resources support the Striving Readers program? 


Toa 
To a small moderate To a large 
Total Not at all extent extent extent Don’t know 


0 
PT aie | ae | 31a | 43.0% 


Do you have a school librarian? 


P12 77.7% 22.3% 


Of those who have a school librarian... 


How does the librarian work with you? (Check all that apply.) 


PI POrs 
Checked Checked 


8 1 
54.3% 45.7% 
3 6. 


Other ways teachers work with librarians. 


I'm an ESL pull-out teacher. 
Other (please | individual assistance to students 


does the as : F ; 
iyearian work She also teach skills in class using the resources in the library. 


with you? she has her own plan and work that she wants them to do 


we have worked together on certain units 
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School Library 


To what extent does the librarian consult with classroom teachers in using Striving Readers 
library funds to order reading materials that are grade level and content appropriate? 


Toa 
To a small moderate To a large 
Total Not at all extent extent extent Don’t know 


2) 


How does the librarian work with your students? (Check all that apply.) 


Not 
Total Checked Checked 


Does not work with my students 
Works with students on research skills 
Directs students to resources tied to curriculum 


Other ways the librarian works with students 


Book club 
I'm not sure/Don't Know 


Other (please | / am sure she works with my ELL students when they visit the Library 
specify): How  |_With their classroom. 


does the Iam unsure a possible study hall atmosphere 


librarian work | Offers extra reading practice. 


with your 
y Shows students how to respond to text. 
students? 


Though our middle grades students do not have an assigned library 
period, our school librarian is extremely helpful in providing my 
homeroom with ample time to borrow books and do research 
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Collaboration with Literacy Intervention Teacher (LIT) 


How often do you meet or collaborate with the LIT in the following settings? 


Less than 
oncea 1- 
Total month 
N % 


Scheduled one-on-one meetings 119 . 16.0% 


Impromptu one-on-one meetings (during io 
lunch, prep periods, before/after school, etc.) ae : ina 


3timesa | 1- 
month 


% 
34.5% 


25.4% 
36.7% 


3 times a 
week 


% 
29.4% 


26.3% 
41.7% 


4-5 times a 
week 
% 
5.0% 


23.7% 
3.3% 


Grade-level team meetings 120 : 11.7% 
Literacy team meetings 119 : 7.6% 


58.8% 


21.8% 


2.5% 


To what extent has your collaboration with the LIT facilitated your efforts to use the following 
methods to support struggling readers in your class? 


To asmall 
Sa pares at all peg 


Differentiating instruction oo - ver 38.8% 
| 119 i 17.6% 31.1% 40.3% 


Scaffolding of instruction 


Toa 


moderate 


extent 


To a large 
extent 


% 


Student groupings 17.5% 29.2% 45.0% 


20.0% 36.7% 


24.2% 


Using the media center 120 19.2% 
Using listening cneters 119 22.7% 


21.8% 34.5% 


Using handheld computers 27.7% 20.2% 26.9% 25.2% 
Using assessment data to monitor student progress 21.0% 24.4% 50.4% 
Using student assessment data for instructional planning 20.0% 25.8% 47.5% 


To what extent has your collaboration with the LIT facilitated your ability to 
provide effective instruction in the following areas for struggling readers? 


Toa 
To a small moderate 
a at all extent sad 


To a large 
aa 


50% 
10% a 0% 


6.7% 
36.196 
23.7% 
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Collaboration with Literacy Intervention Teacher (LIT) 


Overall, how effective has the literacy intervention teacher (LIT) push-in 
been in improving the reading skills of struggling readers in your classroom? 


Total effective effective effective Effective effective 


Pf 0 [75% | 142% | 175% | 30.0% | 30.8% 
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Professional Development 


Did you participate in the following Striving Readers professional 
development sessions conducted during the 2008-2009 school year? 


i 
ee 
107 3 6. 


School-based professional development | 107 | 83.2% =| 16.8% | 


Yes 
% 


If you participated, how useful were the following Striving Readers professional development 
sessions conducted during the 2008-2009 school year? 


Total Not Useful Useful Useful Useful 
| 66 | 0% | 16.7% | 48.9% 
39.1% 


Increasing student motivation 
Supporting students' self-directed learning 
sing before, during, and after reading strategies and techniques 


34.3% 

3 

sing student assessments to guide and inform instruction 

sing literacy-based software 28.3% 71.7% 
sing the PRC2 model | 108 | 71.3% 28.7% 


Or 
Creating literacy-rich classroom environments 6 
85.0% 15.0% 
6 
46.7% 53.3% 
3 
74.1% 25.9% 
2 
i 1 
Using the whole-part-whole classroom instruction model | 105 71.4% 


Building academic vocabulary , 
Differentiating instruction : 
ici 69.4% 30.6% 
i 47.7% 52.3% 


U 
U 
Using handheld computers (Palm Pilots) for teaching and learning 
U 
U 


Did you receive professional development addressing this topic? 
Classroom libraries | 109 
| 108 | 65 
Explicit vocabulary instruction | 108 | 
incorporating extsess———SSSCSC~S~S~S~S~Td 8 | 
| 108 | 68.5% 
| 107 | 70 
8 


N 
108 
109 
108 
107 
108 
108 
107 
107 
108 
108 
107 
106 

08 
105 
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Professional Development 


If received, what impact did the professional development have on your comfort with each teaching practice? 


Total No Impact Impact Impact Impact 
10.7% 


Supporting students' self-directed learning = 2.1% 12.5% 50.0% 35.4% 
Using before, during, and after reading strategies and techniques 44.6% 47.3% 
ne student assessments to ene and inform instruction .0% 10.5% 42.1% 47.4% 
Using literacy-based software .0% 10. 0% 43.3% 46.7% 
Using the PRC2 model 3.9% 15. a= 41.6% 39.0% 


Using the whole-part-whole classroom instruction model 40.5% 


re] 
yes | 
yes | 
22.0% 


Metis Associates 2010 
All Treatment Schools: Literacy Teachers 
Page 27 L-56 


Professional Development 


Please check the techniques in the list below for which you would like to 
receive more training. (Check all that apply.) 


Not 
Total Checked Checked 


% 
57.0% 43.0% 
47.1% 52.9% 
60.3% 39.7% 
61.2% 38.8% 
89.3% 10.7% 
66.1% 33.9% 
67.8% 32.2% 
54.5% 45.5% 

1 
6 
1 


Academic Vocabulary for content terms (e.g., Marzano) 
Morphology instruction (e.g., Shane Templeton) 

Word study sorts and concepts (e.g., Donald Bear) 

Words Their Way (e.g., Donald Bear & Shane Templeton) 
K 


Using PRC2 for comprehension instruction 


= 


Using PRC2 for vocabulary development 
Differentiating instruction 

Everybody Reads To (ERT) 

Exclusion Brainstorming 
List-Group-Label 
Predict-Locate-Add-Note (PLAN) 
ReQuest 


Interactive Notation System for Effective Reading and 
Thinking (INSERT) 


Read Aloud/Think Aloud 
ABC Graffiti 
Guided Reading and Summarizing Procedure (GRASP) 


Teaching summarizing as a comprehension strategy 


38.0% 
33.1% 
28.9% 
43.0% 
41.3% 


65.3% 34.7% 


18.2% 
28.1% 
33.196 
36.496 
27.3% 
18.2% 
25.636 
25.636 
36.496 
50.4% 


Teaching questioning as a comprehension strategy 
Teaching predicting as a comprehension strategy 


Teaching text structure as a comprehension strategy 


Teaching visualization as a comprehension strategy 


Teaching inferring as a comprehension strategy 


N 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 
121 


Teaching metacognition as a comprehension strategy 
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Respondent Information 


Primary Role or Teaching Assignment 


General education teacher (self-contained classroom teacher) 14.9% 

English language arts teacher 19.8% 
What is Teach English language arts and other academic subjects 32.2% 
your Bilingual/ELL teacher 
primary role al edncal h a 
a 20.7% 
assignment? | Reading specialist 


100.0% 


Other (please specify): 
What is your primary 
role or teaching 
assignment? 
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Respondent Information 


What subjects do you teach? 


aN 
I teach all subjects. ce ee 
P40 33.1% 
| 98 | 81.0% 
| 90 | 74.4% 
93 
seid 
| tio 


76. — 
33.1% 
505% 


I only teach English language arts | 121 


N 
121 
121 
121 
121 
121 
121 

21 


What other subjects do you teach? 


ee ee ee 
Advisory, Spanish Language Arts EE a eae 
Other: (Please eel one a 
Specify) (In addition [Compurer 91% 
to English language 
arts, what other Special Ed. Upper Grades 
subject areas do you Special Education 18.2% 
teach? ) — 
15.5% 


Experience 


ee 
Flow many years Rave you been teaching? a a 


How many years have you been teaching at this school? a 


How many years have you been teaching reading/English 
language arts? 
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CPS Striving Readers Evaluation: 
Spring 2010 Literacy Improvement Survey Results 


All Control Schools: Literacy Teachers* 


* Respondents may include general education teachers, English language arts teachers, bilingual/ELL 
teachers, special education teachers, reading specialists and/or other teachers who indicated that their 
role included the teaching of English language arts in addition to other subject areas. 


metis associates 


90 Broad Street 
Suite 1200 
New York, NY 10004 
212-425-8833 
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Survey Response Rates: 


scnost_| reponse | permis | "ea 

Overall 79 124 64% 
Aldridge 3 | 300% 
Carnegie 0 9 0% 
Carver* 0 9 0% 
Casals 4 4 100% 
Clark 2 | 200% 
Dubois | 2 50% 
Copernicus** 0 7 0% 
Dvorak 0 7 0% 
Emmet 2 2 100% 
Esmond 0 0 ha 
Gregory | 3 33% 
Henderson 3 2 150% 
Madison 4 4 100% 
Mann | 7 14% 
McKinley 0 | 0% 
Mireles 7 9 78% 
Morgan | 3 33% 
O'Keefe 3 7 43% 
Otis 6 6 100% 
Parkman 0 3 0% 
Pasteur 9 8 113% 
Pullman 3 | 300% 
Reinberg 8 3 267% 
Schiller*** aS i a 
Sexton | 0 Aa 
South 

Chicago*** = . =; 
SpLy $ 6 100% 
Swift 5 3 167% 
Turner-Drew 3 4 75% 
Wacker 0 3 0% 
Walsh 0 5 0% 
Whistler 6 4 150% 


+ Response rates exceed 100% at some schools where surveys were received from more literacy teachers than were 
indicated in the district's records. 


* Carver Middle School consolidated with Carver Elementary School as of SY09. 
*** No responses were received from Copernicus because it had been identified as a turnaround school as of the 
2009-2010 School Year. In order to reflect the original random selection of participating schools, however, it is still 


included in the calculation of response rates. 


** Schiller and South Chicago, two of the original Striving Readers schools, were not included in this survey 
administration because they closed after SY09. 
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Comprehensive Instruction 


In a typical classroom, how often do you use the following practices to help students increase reading comprehension? 


Explicit instruction in the use of any one or more of the following 
comprehension strategies: summarizing, questioning, predicting, text 
structure visualization, inferring and metacognition 


Not 
Familiar 


Less than 
oncea 
month 


1-3 times a 
month 


1-3 times a 
week 


4-5 times a 
week 


% 


% 


% 


% 


% 


Establishing the purpose for reading. 


Monitoring students' comprehension through questioning. 


Making connections to background knowledge. 


Making connections between texts. 


Synthesizing information within text or across texts. 


Using differentiated instruction (i.e. providing different content, 
resources and/or instructional techniques specifically tailored to meet 
students’ individual educational needs and/or learning styles) 


Use of before, during, and after (BDA) reading strategies for 
comprehension instruction (A student constructed mental framework 
for reading begun before reading even begins, strengthened as students 
interact with the text during the reading, and reflected 


Using partner reading strategies for comprehension instruction 


In a typical classroom, how often do you use the following practices to help students build their vocabulary knowledge? 


Not 
Familiar 


Less than 
oncea 
month 


1-3 times a 
month 


1-3 times a 
week 


4-5 times a 
week 


Explicit instruction in vocabulary 


% 


% 


% 


% 


% 


Modeling the use of word parts 


Review of vocabulary words 
Use of vocabulary notebooks 


Use of partner reading strategies for vocabulary development 


Use of before, during, after (BDA) reading strategies for 
vocabulary instruction 


Words Their Way 


Academic vocabulary for content terms (e.g., Marzano) 


Word study sorts and concepts (e.g., Donald Bear) 


Not 
Familiar 


13.2% 


7.9% 18.4% 5.3% 6.6% 


Less than 
oncea 
month 


1-3 times a 
month 


1-3 times a 
week 


4-5 times a 
week 


Teacher read aloud 


% 


% 


% 


% 


% 


Teacher interactive read aloud 


Shared reading (students and teacher take turns in reading) 


Modeling reading for students 


Explicit instruction in guided oral reading 


Students listen to audio books/play aways 
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Comprehensive Instruction 


In a typical classroom, how often do you use the following techniques to help students develop better reading strategies 
and skills? 


Less than 
Not oncea 1-3 timesa | 4-5 timesa 
Familiar month week week 


% % % % 


Everybody Reads Together (ERT) 


Exclusion brainstorming 


List-Group-Label 
Predict-Locate-Add-Note (PLAN) 
ReQuest 


Interactive Notation System for Effective Reading and 
Thinking (INSERT) 


ABC Graffiti 
Guided Reading and Summarizing Procedure (GRASP) 
KWL 


Less than 
oncea 1-3 timesa | 4-5timesa 
week 
29.1% | 29.1% | 35.4% 
26.6% 30.4% 39.2% 
20.3% | 26.6% | 49.4% 


Considering your own instruction (not that of literacy support staff or other 
instructors in your classroom), in a typical classroom, how often do you apply 
differentiated instruction in your classroom? 


Almost 
About half | Most of the | every lesson 
Total Never Occasionall the time 


Pom | 0% |i | 10.3% | 295% | 48.7% 
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Purposeful Assessment 


Indicate whether you use the data from the following assessments. (Please check all that apply.) 


Spelling Inventories 


Other Assessments 


point 
2 
cloze procedure- open and forced choice 
English Language Proficiency 
Other ISAT 
City 
specify): Ww 


Of those who are using Reading Benchmark Assessments, indicate how you use this assessment 


Not using for this purpose 

Se Re es a Se ee 
[Saeenme | 72 | 6 | 033% | 12 | 167% | 
7m [| 36 | 50.0% [| 36 | 50.0% 
72 | 35 | a8e% =| 37 | 51.4% 
m2 [12 | 16.7% [| 00 | 3.3% 
72 


N 
36 
35 
12 

7 
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Purposeful Assessment 


Of those who are using the Illinois Standards Achievement Test, indicate how you use this assessment 


77 
Benchmarking 45 58.4% 
Progress Monitoring 77 42 54.5% 


33 
33 
33 
33 


Not using for this purpose 
76 
76 | 60 | 78.9% | 
76 
76 


Not using for this purpose 
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Purposeful Assessment 


Of those who are using Spelling Inventories, indicate how you use these assessments 


Not using for this purpose 
95. 5. 


2 | 8 
0 
40 8 
3 
4 ae al 


40 75.0% 
1 


4 [13 32.5% 67.506 
20 85.096 15.0% 
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Data Driven Instruction 


Indicate the extent to which you use student assessment data for each of the following purposes 


Toa 
To Some Moderate | Toa Large 
Not at All Extent Extent Extent 


% 
10.1% | 4.8% 


%o 
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Grade-Level Teams 


Do you currently have grade-level teams at your school? 


———————e— 


[ vatian | _% 
P| 94.9% 51% 


(Of those schools with grade-level teams): Overall, rate the grade-level team’s ability to use classroom 
assessment data in the following ways. 


Address the literacy needs of all students 
Address the needs of struggling readers 
Formalize lesson plans 


Identify students eligible for targeted interventions 33.8% 56.8% 


Identify strengths 29.3% 60.0% 
Identify teaching and learning strategies 


Improve classroom practice 
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Literacy Teams 
Do you currently have vertical literacy teams at your school? 


% 


P73 9.6% | 30.496 


(Of those schools with literacy teams): Overall, rate the quality of the literacy team’s performance in the 
following areas. 


Good __| Excellent | Not Sure 
Valid N % 


Using assessment data to pinpoint the staff's pd needs 5 1.8% 18.2% 27.3% 49.1% 3.6% 
Addressing the needs of all students 5 5% 14.5% 34.5% 43.6% 1.8% 


5 3.6% 10.9% 32.7% 49.1% 
16.7% | 27.8% [48.1% 


5 : 
5 5. 
Addressing the needs of struggling readers 
5 ; 
4 3. 
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School-wide Intervention Materials 


Indicate how frequently you currently use the following materials to teach literacy. 


Listening Centers 


N/A (Do 
Not Have 


Not 
Currently 
Using 


16.9% 
21.8% 


15.6% 
6.4% 


ae ees 


Less than 
oncea 
month 


4 to 5 times 
a week a week 


a month 
% % % 


10.4% 15.6% 24.7% 16.9% 


1 to 3 times 


Media Centers 
Classroom Libraries 


2.6% 
14.9% 


0% 
12.2% 


5.1% 17.9% 25.6% 23.1% 
0% 19.7% 


Vocabulary Notebooks 
Reading Response Notebooks 


74 
76 
School library 6 


19.7% 13.2% 


10.5% 


27.6% 50.0% 
6.8% 5.4% 
1.3% 7.9% 


28.9% 28.9% 
39.5% 10.5% 


Reading Anthologies 


7 11.8% 
7 28.6% 
7 : 


14.3% 


32.4% 28.4% 
6.6% 21.1% 
5.2% 11.7% 


16.9% 23.4% 


vA 
6 | 145% | 13.2% 27.6% | 40.8% 
0 


Other informational texts (other than text sets) 7 


5.7% 


18.6% 38.6% 28.6% 


If using, rate how comfortable you are with using these materials to support student learning in language arts. 


Not at all 
comfortable 


Very 
comfortable 


Media Centers 


% 


38 ; 
36 : 


( % 


% 


Classroom Libraries 


30 


U 
U 
U Vocabulary Notebooks 


Reading Response Notebooks 


Using 


2 


9 : 
26 ; 


0% 
School library 24 .0% 
Reading Anthologies 32 9.4% 


23.1% 
20.8% 
15.6% 
13.6% 


Using 


Vocabulary Development 

Fluency 

Reading Comprehension 

Writing Skills 

Word Parts 

Word Recognition 

Spelling 

Grammar 

To teach content themes 

To develop students' self-directed learning 


To supplement students' textbook reading 


Reading Basals 22 9.1% 0% 
Other informational texts (other than text sets) 23 0% 0% 


13.0% 


79.6% 
87.8% 
24.5% 
10.2% 
42.9% 
12.2% 
24.5% 
36.796 
69.4% 
73.5% 


77.6% 


53.1% 


School-wide Intervention Materials 


Of those who are using each Media Centers, indicate how you are using this material/resource 


a 


Vocabulary Development 55.7% 44.3% 
59.0% 41.0% 
5 4 


Reading Comprehension 34.4% 65.6% 


78.796 21.396 
67.2% 32.8% 


5 4 
0 9 
0 39 
75 4 


Writing Skills 


65.6% 


54.1% 


34.4% 


45.9% 


organizational features or expository writing 


To activate students' prior knowledge 


Of those who are using a Classroom Library, indicate how you are using this material/resource 


? 
i 
| 
| 


11.7% 


Reading Comprehension 
Writing Skills 
Word Parts 


Word Recognition 


72.7% 


58.4% 


27.3% 


41.6% 


Grammar 


To teach content themes 


To activate students' prior knowledge 


1 58. 
| 
ae 7a oe 6s | 88, 
3 6 
| 
| 
Spelling 
| 
3 6 
; 
| 
2 7 
| 
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School-wide Intervention Materials 


Of those who are using Vocabulary Notebooks, indicate how you use this material/resource 


fo 


7% 


75.4% 


3% 


24.6% 


Vocabulary Development 


Reading Comprehension 54.1% 45.9% 


Word Recognition 


19 80 

i 

7 52 
3 6 
70 9 


55.7% 44.3% 


3 
organizational features or expository writing 


To activate students' prior knowledge 


68.9% 


55.7% 


31.1% 


44.3% 


Of those who are using Reading Anthologies, indicate how you use this material/resource 


13.7% 86.3% 


Reading Comprehension 
Writing Skills 
Word Parts 


72.5% 
68.6% 


27.5% 
31.4% 


Word Recognition 


Grammar 


51.0% 


47.1% 


49.0% 


52.9% 


5 
6 


To teach content themes 


To activate students' prior knowledge 


7 2 

62 7 
Spelling 

33 6 

8 
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School-wide Intervention Materials 


Of those who are using Reading Basals, indicate how you use this material/resource 


Soe 


4% 


22.4% 


77.6% 


77.6% 


Vocabulary Development 


Reading Comprehension 91.4% 


58.694 


Word Recognition 


22 ? 
1 
; 
| 


43.1% 


32.8% 


53.4% 


7 
51.7% 
7 


To supplement students' textbook reading 
To develop students' self-directed learning 


' 67.2% 
Teaching students to identify and use text structure 


56.9% 


67.2% 


Teaching students to identify and use the 
organizational features or expository writing 


To activate students' prior knowledge 


N 
58 
58 
58 
58 
58 
58 
58 
58 
58 
58 
58 
58 
58 

8 


Of those who are using the School Library, indicate how you use this material/resource 


| 
| 
| 
i 8 
: 2 


Vocabulary Development 
Fluency 
Reading Comprehension 
Writing Skills 
Word Parts 
Word Recognition 
Spelling 

Grammar 

7 


29.8% 


49.1% 


64 

56.1% 
0 

6 


To teach content themes 
45.6% 
35.1% 


1 
57.9% 


70.2% 


50.9% 


To supplement students' textbook reading 
To develop students' self-directed learning 
Teaching students to identify and use text structure 


Teaching students to identify and use the 
organizational features or expository writing 


To activate students' prior knowledge 


3. 
| 
: 
i 
: 
| 
re ne? 
38 
; 
: 
: 
. 
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School-wide Intervention Materials 


Indicate whether you use each of these materials in your classroom 
to support grouping strategies or differentiated instruction. 


0 
Other informational texts 


Of those using Listening Centers, indicate how you use this material/resource to 
support grouping strategies or differentiation 


Seen rane 
whale Ceaarge ap 7 : 


Whole Chass/Large Group 
[Small Group/Pas———S*dtCSC*C‘iOSCdYSSC« C=C ad —~~—C«.0% 
50 
a 


Individual Work | 650. 58.0% 


Support Differentiated Instruction | 50. 


Of those using Media Centers, indicate how you use this material/resource to 
support grouping strategies or differentiation 


pean 
Thole Care ap 7 


Whole CasfLarge Group 
62 


Support Differentiated Instruction | 62 | 


9 
38.7% 
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School-wide Intervention Materials 


Of those using Text Sets, indicate how you use this material/resource to support 
grouping strategies or differentiation 


Whole Class/Large Group 


A 
[Small Group/Pais «+66 20+ 30.3% | 46 | 69.7% _| 
individual Work | _66__| 32 | 48.5% | 34 | 51.5% _| 
Suppor Differentiated insmucion | 66 | 24 | 36.4% | 42 | 63.6% | 


Of those using a Classroom Library, indicate how you use this material/resource 
to support grouping strategies or differentiation 


Of those using Vocabulary Notebooks, indicate how you use this 
material/resource to support grouping strategies or differentiation 


Not Using for this purpose 


44.3% 


Of those using Textbooks, indicate how you use this material/resource to support 
grouping strategies or differentiation 


Not Using for this purpose 


Whole Class/Large Group 11.0% 


35.6% 


1 
Individual Work 
38.4% 


Support Differentiated Instruction 


| 73 | 
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School-wide Intervention Materials 


Of those using Reading Response Notebooks, indicate how you use this 
material/resource to support grouping strategies or differentiation 


Whole Class/Large Group 


ZO 
[Small Group/Pais ———<d ~~ is iam | a5 ~~. | 
Finividual Work —*|_60_| 17 | 28.3% | 43 | 71.7% _| 
Suppor Differentiated insmucion | 60 | 25 | a7 | 38 | 58.3% | 


Of those using the School Library, indicate how you use this material/resource to 
support grouping strategies or differentiation 


Of those using Reading Anthologies, indicate how you use this material/resource 
to support grouping strategies or differentiation 


| 7 
53 | 8 


39.6% 60.4% 
Support Differentiated Instruction | 53) | 


41.5% 


Of those using Reading Basals, indicate how you use this resource/material to 
support grouping strategies or differentiation 


59 
59 


Small Group/ Pairs 
Individual Work | 59 
Support Differentiated Instruction | 59 
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School-wide Intervention Materials 


Of those using Other Informational Texts (other than text sets), indicate how you 
use this material/resource to support grouping strategies or differentiation 


Not Using for this purpose 
N 
46.5% 
45. 


Whole Class/Large Group 


Individual Work 71 
Support Differentiated Instruction 71 43.7% 


[Small Group/ Pais —————~+| Ti 
1 


L-7ipage 16 


Use of Classroom Computers for Literacy Instruction 


Do you use classroom computers (desktop, laptop, or handheld computers) to teach literacy? 


[ES ESS) 


% % 
P79 | 79.7% | 20.396 


If you are NOT yet using classroom computers to teach literacy, please indicate 
why you are not using them. 


Not 
Total Selected 
N 


56.3% 
68.8% 


We do not have computers in the classroom 


Some or all of the computers are not working properly 


We do not have enough computers for every student and I do not 
like to have some students use them while others can not 

I do not feel that they offer sufficient added benefit compared to 
traditional media (e.g., print, paper and pencil) to be worth the 
trouble 


93.8% 6.3% 


TOO 


(Of those using classroom computers): In a typical classroom, how 
often do students use classroom computers during literacy instruction? 


Less than 
Total month month week week 
Peet 3% | 25.8% | 40.3% | 22.6% 


(Of those using classroom computers): Rate how comfortable you are with 
using the classroom computers to support your literacy instruction. 


Total Comfortable 2 3 Comfortable 


SN A 
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Use of Classroom Computers for Literacy Instruction 


(Of those using classroom computers): Specific academic foci or 
instructional objectives supported by the use of classroom computers 


aa 
Total Checked 

N 
65.1% 
71.4% 
71.4% 
39.7% 
39.7% 


Developing students' reading comprehension strategies 
Writing skills 

Word parts 

Word recognition 


47.6% 
92.1% 
61.9% 
60.3% 
77.8% 
50.8% 


To activate students' prior knowledge 27.0% 73.0% 


Grammar 

Locating information 
Evaluating information 
Synthesizing information 
Organizing information 


Demonstrate knowledge of key concepts 


Spelling | 663s] 65.1% 34.9% 


63 
63 
63 
63 
63 
63 
63 
63 
63 
63 
63 
63 
63 
63 
63 
3 


(Of those using classroom computers): Which instructional practices and grouping 
structures do you support through the use of classroom computers during literacy lessons? 


are 
Total Checked Check 
N 


Small group/ Pairs 
Individual Work 
Monitoring distribution and completion of assignments 


Assessing students' literacy skills 


30.2% 69.8% 


44.4% 55.6% 


Book club dicussions 


Differentiating instruction for struggling readers 


Differentiating instruction for English language 
learners/special education students 


tia 
Monitoring students' progress 
3 
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Classroom Library 


The ways you use your classroom libraries 


Not 
ee 
[Forcontentarea instruction —__| : 40. 


For content area instruction 
9 


I do not have a classroom library 93.7% 


How do you use Interest Inventories? 


Do you use interest inventories to help 
students self select reading material? 
Do you use interest inventories to guide your 
y oo 78 61.5% 38.5% 
purchases for the classroom library? 


Please indicate how true each of the following statements are about the organization of books in your 
classroom library 


Not At All Slightly Somewhat 
Total True True True Very True 
N % % % % 

My classroom library is easily accessible to students 
72 0% | 42% | 36.1% | 59.794 
73 | 14% | 11.0% | 19.2% | 68.5% 


My classroom library includes a variety of reading materials that 
are appropriate for readers of differing abilities 

My classroom library includes a variety of texts that appeal to 
readers with different interests 


My classroom library reading materials grouped by genre 


My classroom library has reading materials clearly labeled 


My classroom library includes both nonfiction and fiction 


Are you able to consider students needs and reading abilities when ordering reading material 
for your classroom library? 


Toa 
To a small moderate To a large 
Total Not at all extent extent extent Don’t know 


Students' needs and reading abilities 12.3% 26.0% 58.9% 
32.4% [54.1% 
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School Library 


In a typical classroom, how often do you take your class to the library? 


Sometimes 
Rarely (less (at least Often (at Almost 
than once a least once a daily or 


ee 06.2% 


To what extent do the library resources support your schools literacy goals? 


Toa 
To a small moderate To a large 
Total Not at all extent extent extent Don’t know 


0 0 0 
P75 | 32% | 16.4% | 32.9% | 26.3% 


Do you have a school librarian? 


Not 
ae rae 
The librarian does not work with me 


The librarian provides resources for class projects 42. Toe 57, at 
The librarian and I collaborate on how to supplement i 

6.2% 3.8% 
lessons with library resources 


Other ways teachers work with librarians. 
He is there for me when I need him 16. on 


Other (please | I ask the librarian for supplimental books 
specify): Is assigned to work w/ the class 16. = 


16. a 


(How does — | 
ea Librarians teach a curriculum and support my projects when 

work with requested. 

you?) Library is used when classrooms are assigned for prep periods 


She teaches library 
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School Library 


(Of those with a school librarian): To what extent does the librarian consult with classroom 
teachers to order reading materials that are grade level and content appropriate? 


Toa 
To a small moderate To a large 
Total Not at all extent extent extent Don’t know 


a a a 


(Of those with a school librarian): How does the librarian work with your students? 
(Check all that apply.) 


ee eee 
Total Checked Checked 


Does not work with my students 90.5% 
Works with students on research skills 34.9% 65.1% 
Directs students to resources tied to curriculum 52.4% 47.6% 


Conducts read-alouds 55.6% 44.4% 
Provides students with information about extracurricular academic 


activities (e.g., spelling bee, writing competitions, events). As 


Other ways the librarian works with students 


a 
Book clubs 20.0% 
Other (please specify): 40.0% 


(How does the librarian Our 8th grade students did not have school hour access to 
work with your students? | the library this year due to scheduling issues. Our librarian 1 20.0% 
(Check all that apply.)) helps run an after-school teen book club. 


L-82page 21 


Collaboration with Literacy Support Staff 


Does your school have a literacy enrichment specialist 
such as a Reading Specialist or Literacy Coach? 


aa ee 


% % 
P79 | art | 32.9% 


(Of those with a literacy enrichment specialist): Please indicate title of the 
literacy enrichment specialist. 


Please 
indicate 
this 
person’s 
title. Literacy team 


Literacy Coordinator 


Literacy Teacher 


Reading Specialist 
We have both [a Reading Specialist and Literacy Coach]. 


Not specified 


How often do you meet or collaborate with the literacy enrichment specialist in the 
following settings? 


Less than 
oncea 1-3 timesa | 1-3timesa | 4-5 timesa 
Total Never month month week week 
Scheduled one-on-one meetings 21.6% 19.6% 19.6% 35.3% 
Grade-level team meetings 13.5% 34.6% 46.2% 
Literacy team meetings 20.0% 22.0% 28.0% 28.0% 


| 
0 
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Collaboration with Literacy Support Staff 


To what extent has your collaboration with the literacy enrichment specialist facilitated 
your efforts to use the following methods to support struggling readers in your class? 


Toa 
To a small moderate To a large 
Not at all extent extent extent 


Differentiating instruction 0 

Scaffolding of instruction 11.5% 26.9% 28.8% 

Student groupings 5 23.5% 19.6% 

Using the media center 5 13.7% 
7 


7% 23.1% 8% 


7 

7 
Using listening centers 42.3% 25.0% 
Using handheld computers 60.8% 
Using assessment data to monitor progress 13.5% 30.8% 


Using assessment data for instruction planning 13.2% 28.3% 50.9% 


To what extent has your collaboration with the literacy enrichment specialist facilitated 
your ability to provide effective instruction in the following areas for struggling readers? 


Toa 
To a small moderate To a large 
Total Not at all extent extent extent 


Comprehension 


Z 1 
Writing skills 14.0% 
24 2 


Overall, how effective has the literacy enrichment specialist been in 
improving the reading skills of struggling readers in your classroom? 


Total effective effective effective Effective effective 


%o 6 ( ( ( 
PT 2a | 3% | 250% | aoa | 73% 
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Professional Development 


Did you receive professional development addressing this topic? 


sing whole-part-whole model 


Yes, 
participated in 
PD. 
% 


No, did not 


43.2% 56.8% 
4 
4 
4 


pa 
4 
5 
5 
5 
4 
4 


cipate. 
% 


rti 

28. 71 
54. 5 
89. 11 
3. 6 
35. 64 
84. 15 
32. 67 
25. 74 
51. 8 
38. 61 


If received, what impact did the professional development have on your comfort with each teaching practice? 


partici 


pa 


ed in PD. 


Building academic vocabulary 


Total No Impact 


Slight 
Impact 


Moderate 
Impact 


Major 
Impact 


Valid N % 


6.3% 


62.5% 


31.3% 


partici 
partici 


pa 


pa 


ed in PD. 
ed in PD. 


Classroom libraries 


Creating literacy-rich environment 


32 0% 
20 5.0% 
9 


3 


15.0% 


40.0% 


40.0% 


partici 


pa 


ed in PD. 


Differentiating instruction 


60 


partici 


pa 


ed in PD. 


Explicit vocabulary instruction 


30 


partici 


pa 


ed in PD. 


Incorporating text sets in your instruction 


25 


partici 


pa 


ed in PD. 


Increasing student motivation 


31 


partici 
partici 


pa 


pa 


ed in PD. 
ed in PD. 


Supporting students’ self-directed learning 
Using before, during, after reading strategies and techniques 


28 


partici 


pa 


ed in PD. 


Student assesments to guide instruction 


partici 


pa 


ed in PD. 


Using classroom computers 


partici 


pa 


ed in PD. 


Using literacy-based software 


partici 


pa 


ed in PD. 


Using partner reading 


partici 


pa 


ed in PD. 


Using whole-part-whole model 
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Struggling Readers: Extended Day (Afterschool) Intervention 


Does your school currently offer after or before school 
programming specifically targeting struggling readers? 


tf 


es A 


How many of your current students are involved in the after or before school program? 
| | iat} Nne_}_ttes tte 7 arama | 
T7706 | 0% | mie | 50% | a3 


Overall, how effective has the after or before school component been in improving the literacy 
abilities of struggling readers? 


cael ae aes aa ee aa ae ane t — 


es 
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RESPONDENT INFORMATION 


Total Checked Checked 


None - I 
only teach 
English Other: 
Social language (Please 
Total Mathematics Science studies : i 
what other subject areas do you teach? 


Other subjects taught 


All Subjects 25.0% 
one re Iam the Reading Specialist 
Specify) (In - 
me 


English language | Math only/some writing 


arts, what other Math, Science, Soc. Studies, Writing 
subject areas do Math, Science, Social Studies 
you teach?) 

16.7% 


L-87page 26 


RESPONDENT INFORMATION 


Primary Role or Teaching Assignment 


TOolhTOOO or 


Other (plese speciy; ——————SSSSSSSSSCSC*d SY 


Other Primary Role or Teaching Assignment 


GS 


Experience 


atic N | Minimum | Maximum | Mean __| 
Flow many years have you been teaching? 
How many years have you been teaching at this school? | 7 |{ 1 | 30 | 98 | 


How many years have you been teaching reading? 
How many years have you been teaching in your subject area? 
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CPS Striving Readers Evaluation: 
Spring 2010 Literacy Improvement Survey Results 


All Treatment Schools: Literacy Intervention Teachers 


Responses were received from the facilitators at 29 of 30 remaining schools®, for a response rate of 97%. 


&Q metis associates 


120 Wall St 
21* floor 
New York, NY 10005 
212-425-8833 


° Abbot closed after school year 2008-2009. In addition, no response was received from Bethune 
because it had been identified as a turnaround school as of the 2009-2010 school year; however, in order 
to reflect the original random selection of participating schools, it is still included in the calculation of 
response rates. 
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Targeted Intervention 


In a typical classroom, how often do you use the following grouping structures? 


Less than 
once a 1-3 timesa | 1-3 timesa | 4-5 timesa Multiple 
month month week week times a day 


% % % % % 
Whole Class/Large group 
Individual Work 


Small groups or Pairs 


Considering your push-in intervention with Tier 2 and 3 students, in a typical classroom, 
how often do you apply differentiated instruction? 


Almost 
About half | Most of the | every lesson 
Occasionally the time time or activity 


Valid N % % % % 


During your work in the regular classroom with Tier 2 and 3 students, in a typical 
classroom, how often do you use the following practices to help struggling readers increase 
reading comprehension? 


Less than 
oncea | 1-3 times | 1-3 times | 4-5 times 
month a month a week a week 


% % % % 
Explicit instruction in the use of any one or more of the 
comprehension strategies: summarizing, questioning, 
predicting, text structure visualization, inferring and 
metacognition 


Establishing the purpose for reading 


Monitoring students' comprehension through questioning 


Making connections to background knowledge 


Making connections between texts 


Synthesizing information within text or across texts 


Using differentiated instruction 


Use of before, during, and after (BDA) reading strategies 
for comprehension instruction 


Using Pairing for Partner Reading & Content Too 
(PRC2) for comprehension instruction 
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Targeted Intervention 


During your work in the regular classroom with Tier 2 and 3 students, in a typical 
classroom, how often do you use the following practices to help struggling readers build 
their vocabulary knowledge? 


Less than 
oncea 1-3 times | 1-3 times | 4-5 times 
month a month a week a week 


% % % % 
Explicit instruction in vocabulary 


Modeling the use of word parts 


Review of vocabulary words 


Use of vocabulary notebooks 


Use of PRC2 for vocabulary development 

Use of before, during, and after (BDA) reading 
strategies for vocabulary instruction 

Words Their Way 


Academic Vocabulary for content terms (e.g., 
Marzano) 

Word study- word sorts and concepts (e.g., Donald 
Bear) 


Morphology instruction (e.g., Shane Templeton) 


During your work in the regular classroom with Tier 2 and 3 students, in a typical 
classroom, how often do you use the following practices to help struggling readers develop 
fluency? 


Less than 
once a 1-3 times 4-5 times 
month a month a week 


% % % 
Teacher read aloud 


Teacher interactive read aloud 


Shared reading (students and teacher take turns in 
reading) 


Modeling reading for students 


Focusing instruction on proper and meaningful 
phrasing 


Students listen to audio books, play aways 


The gradual release model (Leading students from modeled instruction to shared instruction to 
guided practice and finally students' independent practice) and explicit instruction in guided 
reading are intended to be use on an "as needed" basis. 
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Targeted Intervention 


During your work in the regular classroom, in a typical classroom, to what extent do you 
feel you are able to meet your Tier 2 and 3 students individual needs through these 
instructional practices? 


Toa 
moderate 
Not at all extent 


% % 


Use of the gradual release of responsibility model for 
reading comprehension instruction 


Use of the gradual release of responsibilty model to 
build vocabulary 


Use of the gradual release of responsibility model to 
develop fluency 


Explicit instruction in guided oral reading to develop 
fluency 


During your work in the regular classroom with Tier 2 and 3 students, in a typical 
classroom, how often do you use the following techniques to help students develop better 
reading strategies and skills? 


Never/ | Less than 
Not once a 1-3 times | 1-3 times | 4-5 times 
Familiar month a month a week a week 


% % % % % 
Everybody Reads To (ERT) 
Exclusion Brainstorming 
List-Group-Label 
Predict-Locate-Add-Note (PLAN) 
ReQuest 


Interactive Notation System for Effective Reading 
and Thinking (INSERT) 


ABC Graffiti 


Guided Reading and Summarizing Procedure 
(GRASP) 


KWL 
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Targeted Intervention 


How often do you meet with ELA classroom teachers at the following grade levels to 
discuss instruction-related issues regarding your work with students in the Targeted 


intervention group? 


Less than 
once a 
month 


1-3 times a 
month 


1-3 times a 
week 


4-5 times a 
week 


% 


% 


% 


% 


Grade 6 teachers 
Grade 7 teachers 
Grade 8 teachers 


In which setting(s) do you typically meet or collaborate with 6TH GRADE ELA classroom 
teachers? 


Not 
Checked Checked 


% % 


Scheduled one-on-one meetings 82.8% 


Impromptu one-on-one meetings (e.g., during lunch, prep periods, 93.19% 
before/after school, etc.) pea 


Grade-level team meetings (i.e., teams consisting of staff across 
subject areas from the same grade, or in grade level "bands") 


100.0% 


Literacy team meetings (i.e., teams focusing on literacy issues across 93.1% 
grade levels) ee 


In which setting(s) do you typically meet or collaborate with 7TH GRADE ELA classroom 
teachers? 


Not 
Checked Checked 


% % 


Scheduled one-on-one meetings 


Impromptu one-on-one meetings (e.g., during lunch, prep periods, 
before/after school, etc.) 


Grade-level team meetings (i.e., teams consisting of staff across 
subject areas from the same grade, or in grade level "bands") 


Literacy team meetings (i.e., teams focusing on literacy issues across 
grade levels) 
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Targeted Intervention 


In which setting(s) do you typically meet or collaborate with 8TH GRADE ELA classroom 
teachers? 


Not 
Checked Checked 


% % 


Scheduled one-on-one meetings 


Impromptu one-on-one meetings (e.g., during lunch, prep periods, 
before/after school, etc.) 


Grade-level team meetings (i.e., teams consisting of staff across 
subject areas from the same grade, or in grade level "bands") 


Literacy team meetings (i.e., teams focusing on literacy issues across 
grade levels) 


On average, how often do you meet with each SIXTH-GRADE classroom teacher to discuss 
implementing each of the following instructional methods for students in the in-class 
Targeted Intervention group? 


Less than 
once a 1-3 times 
month a month 


% % 
Differentiated instruction 


Student groupings 


Use of Striving Readers texts sets, text set teacher 
guides, technology, classroom library, school library 


Use of specific Striving Readers comprehension 
strategies for reading 


Using specific Striving Readers instructional 
techniques for comprehension instruction 


Using specific Striving Readers instructional 
techniques for vocabulary instruction 


Using specific Striving Readers instructional 
techniques for fluency instruction 


Discussing specific students' reading progress 


Coordinating instruction between lessons for the 
whole class and lessons for the Targeted 
Intervention group 

Using student assessment data for instructional 
planning 
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Targeted Intervention 


Data Driven Instruction 


Please indicate the extent to which you use student assessment data for each of the 
following purposes related to your work with students in the Targeted intervention group? 


Differentiating instruction 


Not at All 


To some 
extent 


Toa 
moderate 
extent 


To a large 
extent 


% 


% 


% 


% 


Identifying skills that need to be taught or retaught 


Monitoring student reading progress 


Creating instructional groups (in-class) 
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Targeted Intervention 
School-Wide Intervention Materials 


Indicate, in a typical classroom, how often do you use the following materials to teach 
literacy? 


Not 
Currently 
Using 


% 


Listening centers (classroom CD & Cassette 
Player, Read-Along audio books, playaways, 
headphones) 

Media centers (three computers and a 


printer) 


Classroom library 


Vocabulary notebooks 


Reading response notebooks 
School Library 
Reading anthologies 


Reading basals 


Other informational texts (other than text 
sets) 


If using, rate how comfortable you are with using these materials to support student 
learning in language arts. 


1 - Not at all 5 - Very 
Comfortable Comfortable 


% % 
Listening centers (classroom CD & 
Cassette Player, Read-Along audio 
books, playaways, headphones) 
Media centers (three computers and a 
printer) 


Classroom library 


Vocabulary notebooks 


Reading response notebooks 


School Library 


Reading anthologies 


Reading basals 


U 
U 
U 
U 
U 
U 
U 
U 


Other informational texts (other than text 
sets) 
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Targeted Intervention 
Handheld Computers 


Do you use handheld computers (Palm Pilots) to teach literacy during instruction of Tier 2 
and 3 students in the regular classroom? 


| eer ieerieeeesr 


es 


Reasons for not using handhelds 


Not 
Checked Checked 


% % 
I have not received the handheld computers 100.0% 


Some or all of the computers are not working properly 80.0% 


Some or all of the necessary software applications have not 
been installed on the computers 


80.0% 


I have not received sufficient professional development to feel 
comfortable using them 


100.0% 


Because the Striving Readers program provides only 10 
computers per classroom, and I do not like to have some 100.0% 
students use them while others can not. 


They are being used primarily for whole class instruction 60.0% 


I do not feel that they offer sufficient added benefit compared 
to traditional media (e.g., print, paper and pencil) to be worth 60.0% 
the trouble 


Other (please specify) 60.0% 


Other reasons for not using handhelds. 


They are being used primarily for small group 
instruction. Intervention is provided in those 
small groups. 


Other (please specify): (If you are NOT yet using 


handheld computers, please indicate why you are 
not using them) 


They don't work when I need them. 
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Targeted Intervention 


Handheld Computers 


In a typical classroom, how often do your students use handheld computers (Palm Pilots) 
during Targeted Intervention instruction of Tier 2 and 3 students? 


Less than 
oncea 1-3 timesa | 1-3 timesa | 4-5 timesa 
month month week week 


% % % % 


Rate how comfortable you are with using the Palm Pilots to support your literacy 
instruction during targeted intervention instruction. 


1 Not at all 5 Very 
Comfortable Comfortable 


Which specific academic foci or instructional objectives do you support with the use of 
handheld computers (Palm Pilots) during Targeted Intervention instruction? (Check all 
that apply) 


Not 
Checked Checked 


% % 
Fluency 


Vocabulary development 


Developing strudents' reading comprehension strategies 
Writing skills 
Word parts 


Word recognition 


Spelling 


Grammar 


Organizing information 


Locating information 


Evaluating information 


Synthesizing information 


Demonstrate knowledge of key concepts 


To develop students' self-directed learning 


Teaching students to identify and use the organizational 
features of expository writing 


To activate students’ prior knowledge 


L-98 


Targeted Intervention 


Handheld Computers 


Which instructional methods do you support with the use of handheld computers (Palm 
Pilots) during Targeted Intervention instruction? (Check all that apply) 


Not 
Checked 


Checked 


% 


Monitoring distribution and completion of assignments 


% 


Assessing students! literacy skills 


Monitoring students’ progress 


Teaching comprehension skills 


Using comprehension skills 


Guided reading 


Partner reading 


Individual reading 


Book club discussions 


Please indicate whether you use handheld computers (Palm Pilots) to support grouping 
structures and/or differentiated instruction during instruction of Tier 2 and 3 students in 


the regular classroom. (Check all that apply) 


Not 


Checked Checked 


% 
Whole class/ Large group 


% 


Small group/ Pairs 


Individual Work 


Differentiating instruction for struggling readers 


Differentiating instruction for English language 
learners/special education students 
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Targeted Intervention 
Handheld Computers 


Indicate how frequently your typical Tier 2-3 students use each software application on the 
Palm Pilots during literacy instruction. 


Not Less than 
N/A (Do | Currently | oncea 
Total Not Have) Using month 


Valid N % % % 
iKWL 
Freewrite 
PiCo Maps 


ViewPoint 
Sketchy 


MS Word 

MS Excel 
Slideshow to Go 
Cells 


Internet Browser 


Inspiration 


PAAM management software 
application 


Go Manage 


For those that you are using, rate how comfortable you are with using each software 
application on the Palm Pilots to support your literacy instruction. 


1 - Not at all 5 - Very 
Comfortable Comfortable 


% % 
iK WL 
Freewrite 

PiCo Maps 
ViewPoint 
Sketchy 

MS Word 

MS Excel 
Slideshow to Go 
Cells 


Internet Browser 


Inspiration 


PAAM management software 
application 


U 
U 
U 
U 
U 
U 
U 
U 
U 
U 
U 
U 
U 


Go Manage 
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Targeted Intervention 


Of those who are using each software... 


Indicate how you use iKWL 


Vocabulary development 


Handheld Computers 


Not Using for 
this purpose 


% 
58.8% 


Fluency 


100.0% 


Reading Comprehension 


11.8% 


Writing Skills 


70.6% 


Word parts 


Vocabulary development 


88.2% 


Not Using for 
this purpose 


% 


Fluency 


Reading Comprehension 


Writing Skills 


Word parts 


Vocabulary development 


Not Using 
for this 
purpose 


% 


Fluency 


Reading Comprehension 


Writing Skills 


Word parts 
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Targeted Intervention 
Handheld Computers 


Of those who are using each software... 


Indicate how you use Viewpoint 


Not Using for 
Total this purpose 


Valid N % 
Vocabulary development 83.3% 
Fluency 100.0% 
Reading Comprehension 50.0% 
Writing Skills 100.0% 
100.0% 


Indicate how you use Sketchy 


Not Using for 
this purpose 


% 


Vocabulary development 


Fluency 


Reading Comprehension 
Writing Skills 


Not Using for 
this purpose 


% 


Vocabulary development 


Fluency 


Reading Comprehension 
Writing Skills 
Word parts 
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Targeted Intervention 


Handheld Computers 


Of those who are using each software... 


Indicate how you use Freewrite 


Not Using for 
this purpose 


% 


Vocabulary development 83.3% 
Fluency 100.0% 
Reading Comprehension 83.3% 
Writing Skills 100.0% 


100.0% 


Not Using for 
this purpose 


% 
Vocabulary development 100.0% 


Fluency 85.7% 


Reading Comprehension 42.9% 


Writing Skills 71.4% 


100.0% 


Not Using for 
this purpose 


% 


Vocabulary development 50.0% 


100.0% 


Fluency 


Reading Comprehension 100.0% 


Writing Skills 83.3% 


Word parts 50.0% 


L-103 


Targeted Intervention 


Handheld Computers 


Of those who are using each software... 


Indicate how you use Internet Browser 


Not Using for 
this purpose 


% 


Vocabulary development 


Fluency 


Reading Comprehension 
Writing Skills 


Not Using for 
this purpose 


% 


Vocabulary development 


Fluency 


Reading Comprehension 
Writing Skills 
Word parts 
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Targeted Intervention 


Professional Development 


Did you participate in the following Striving Readers professional development sessions 


conducted during the 2009-2010 school year? 


Did not 
participate 


Participated 


AMP Intensive Intervention Program Training 


% 


% 
82.8% 


2009 Summer Institute 


93.1% 


School-year follow-up institutes 


89.7% 


Bi-weekly LIT training sessions 


96.6% 


Teacher/LIT collaboration 


96.6% 


School-based professional development 


100.0% 


If you participated, how useful were the following Striving Readers professional 
development sessions conducted during the 2009-2010 school year? 


Participated 


AMP Intensive Intervention Program 
Training 


Not Useful 


Somewhat 
Useful 


Moderately 
Useful 


Extremely 
Useful 


% 


% 


% 


% 


Participated 


2009 Summer Institute 


Participated 


School-year follow-up institutes 


Participated 


Bi-weekly LIT training sessions 


Participated 


Teacher/LIT collaboration 


Participated 


School-based professional development 
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Targeted Intervention 


Professional Development 


Did you receive professional development addressing each of the following topics? 


Building academic vocabulary 


Did not 
participate 


Participated 


% 


% 
75.9% 


Clasroom libraries 


51.7% 


Creating literacy-rich classroom environments 


85.7% 


Differentiating instruction 


100.0% 


Explicit vocabulary instruction 


85.7% 


Increasing student motivation 


78.6% 


Supporting students self-directed learning (Gradual release model) 


82.1% 


Using before, during, and after reading strategies 


92.9% 


Using student assessments to guide instruction 


100.0% 


Using handheld computers (Palm Pilots) 


96.4% 


If received, what impact did the professional development have on your comfort with each 
teaching practice? 


Participated 


Building academic vocabulary 


No Impact 


Moderate 
Impact 


% 


% 


Participated 


Classroom libraries 


Participated 


Creating literacy-rich classroom environments 


Participated 


Differentiating instruction 


Participated 


Explicit vocabulary instruction 


Participated 


Increasing student motivation 


Participated 


Supporting students! self-directed learning 
(Gradual release model) 


Participated 


Using before, during, and after reading 
strategies 


Participated 


Using student assessments to guide 
instruction 


Participated 


sing handheld computers (Palm Pilots) 


Participated 


sing literacy-based software 


Participated 


sing the PRC2 model 
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Targeted Intervention 


Professional Development 


Please check the techniques in the list below for which you would like to receive more 


training. (Check all that apply.) 


Academic Vocabulary for content terms (e.g. Marzano) 


Not 
Checked 


Checked 


% 


% 


Morphology instruction (e.g., Shane Templeton) 


Word study-word sorts and concepts (e.g., Donald Bear) 


Words Their Way 


KWL 


Using PRC2 for comprehension instruction 


Using PRC2 for vocabulary development 


Differentiating instruction 


Everybody Reads To (ERT) 


Exclusion Brainstorming 


List-Group-Label 


Predict-Locate-Add-Note (PLAN) 


ReQuest 


Interactive Notation System for Effetive Reading and 
Thinking (INSERT) 


Read Aloud/Think Aloud 


ABC Graffiti 


Guided Reading and Summarizing Procedure (GRASP) 


Teaching summarizing as a comprehension strategy 


Teaching questioning as a comprehension strategy 


Teaching predicting as a comprehension strategy 


Teaching text structure as a comprehension strategy 


Teaching visualization as a comprehension strategy 


Teaching inferring as a comprehension strategy 


Teaching metacognition as a comprehension strategy 
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After-School Program 


In a typical AMP after-school class, how often do you use the following grouping 


structures? 


Whole class/Large group 


Total 


Once per 
class 


2-3 times 
per class 


4 or more 
times per 
class 


Valid N 


% 


% 


% 


Individual Work 


Small groups or Pairs 


In a typical AMP after-school class, how often do you apply differentiated instruction? 


Rarely or 
Never 


Occasionally 
(once or twice a 
week) 


Usually (in 
most 
lessons) 


In every or 
nearly 
every lesson 


In every or 
nearly 
every 

activity 


Valid N 


% 


% 


% 


% 


% 


Do you feel that the AMP after-school program is appropriate to the reading levels of the 


students who are currently participating? 


Not at all 


appropriate 


Somewhat 
appropriate 


Appropriate 


Very 


appropriate 


0, 


ee 


% % % % 
9% 51.7% 34.5% 


Please indicate the proportion of your AMP students for whom the following statements 


are true. 


All or 
almost all 
students 


Most 
students 


Hardly 
any 
students 


A few 


About half | students 


These students should not be in the AMP class 
because their reading levels are too high 


% 


% 


% % % 


These students should nto be in the AMP class 
because their readings levels are too low 


L-108 


After-School Program 


Are there students who are not in the after-school program who should be? 


Yes — Please 
explain: 


0 


() % 
a 


Please explain why there are students who are not in the after-school program who should 


Competition with other after-school programs 
e In other programs (5) 
e One parent wanted the child to be in another program with their previous 
teacher. 
Student chose not to attend 
e _Parent/s does not MAKE student attend 
e They choose not to attend. 
Parent decision 
e Parents did not give permission (5) 
e Some students parents’ pulled them out of the program 
AMP is needed for some Tier 2 students 
e Some students who were classified as tier 2 students were in practice tier 
3 students. 
e Students who were in AMP the previous year and tested out of tier 3 
e They are Tier 2. 


No tier assignment because of missing test scores 
e Students that transfer in after program has started or have not been 
identified as tier 3. 


Transfer student 
e Weren't at our school the previous year, we had no data 
Program capacity limitations 


e Not able to have more than 2 classes paid for. 
e The number would surpass 15 students 
Transportation issues 
Transportation (3) 
One student rides the bus and was unable to stay. 


Two students have been hospitalized for emotional issues. 

Based on ISAT scores—It seems that the majority of students that qualify 
for amps [sic] are in the special ed. category. 

Some Tier 3 students that did not attend 
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After-School Program 


Rate the proportion of your students for whom you think the following statements about 
the AMP after-school program are true. 


The AMP after-school program is.. 


engaging 


Total 


All or 
almost all 
students 


Most 
students 


A few 
students 


Hardly 
any 
students 


Valid N 


% 


% 


% 


% 


The AMP after-school program is... 


interests 


relevant to their 


The AMP after-school program is... 


motivating 


The AMP after-school program is... 


literacy needs 


appropriate to their 


The AMP after-school program is... 


learning style 


appropriate to their 


How comfortable are you with using the Achieving Maximum Potential (AMP) software? 


Not 
comfortable 


Somewhat 
comfortable 


Comfortable 


Very 
comfortable 


0 


P| 28% 


0 % % % 
% 27.6% 31.0% 13.8% 


How frequently do you use the Achieving Maximum Potential (AMP) software for 
struggling readers in the after-school program? 


Less than 
once a 
month 


1-3 times a 
month 


week 


1-3 times a 


4-5 times a 
week 


Multiple 
times a day 


% 


% 


% 


% 


% 
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After-School Program 


If you said that you "never" use the AMP software, please indicate why. (Check all that 
apply). 


Not 
Checked Checked 


% % 
I do not have the computers in the classroom where I teach AMP 90.0% 


The AMP software is not installed on my computers 70.0% 


The computers in the classroom where I teach AMP are not working 80.0% 


I do not believe that the AMP software is effective at building 80.0% 
. 0 


students' literacy skills 
I do not know how to use the AMP software 70.0% 


The reading level of the AMP software is too high for the students oi 
ae 80.0% 
who are currently participating 


The reading level of the AMP software is too low for the students 7 
wea gt 100.0% 
who are currently participating 


Other 80.0% 


Other reason for never using the AMP software. 


AMP Software would not 
install. 

Very time consuming and I 
have to make sure students 
are on task and it's easier to 
use small group instruction to 
monitor students progress. 
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After-School Program 


In a typical AMP after-school class, how often do you use the following practices or 
materials with Tier 3 students to help them increase reading comprehension 


Less than 
oncea 1-3 times 4-5 times 
month a month a week 


% % % 
Explicit instruction in comprehension strategies: 


summarizing, questioning, predicting, text structure, 
visualization, inferring and metacognition 


Establishing the purpose for reading 


Monitoring students' comprehension through 
questioning 


Making connections to background knowledge 


Making connections between texts 


Synthesizing information within text or across texts 


Using differentiated instruction 


Use of before, during, and after (BDA) reading 
strategies for comprehension instruction 


In a typical AMP after-school class, how often do you use the following practices or 
materials with Tier 3 students to help them build their vocabulary knowledge? 


Less than 
once a 1-3 times a | 4-5 times a 
month week week 


% % % 
Explicit instruction in vocabulary 


Modeling the use of word parts 


Review of vocabulary words 


Use of before, during, and after (BDA) 
reading strategies for vocabulary instruction 


Academic vocabulary for content terms 


Word study-word sorts and concepts 


Morphology instruction 
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After-School Program 


In a typical AMP after-school class, how often do you use the following practices or 
materials with Tier 3 students to help them develop fluency? 


Less than 
once a 1-3 times 4-5 times 
month a month a week 


% % % 
Teacher read aloud 


Teacher interactive read aloud 


Shared reading (students and teacher take turns in 
reading) 


Modeling reading for students 


Explicit instruction in guided oral reading 


Focusing instruction on proper and meaningful 
phrasing 


Students listen to audio books, play aways 


The gradual release model (Leading students from modeled instruction to shared instruction to 
guided practice and finally students' independent practice) and explicit instruction in guided 
reading are intended to be use on an "as needed" basis. 


During your work with students in the AMP after-school program, in a typical classroom, 
to what extent do you feel you are able to meet your Tier 3 students individual needs 
through these instructional practices? 


Toa 
moderate 
Not at all extent 


% % 
Use of the gradual release of responsibility model for 
reading comprehension instruction 


Use of the gradual release of responsibility model to build 
vocabulary 

Use of the gradual release of responsiblity model to 
develop fluency 

Explicit instruction in guided oral reading to develop 
fluency 
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After-School Program 
Assessment Data 


Indicate the extent to which you use student assessment data for each of the following 
purposes within the AMP after-school program. 


Toa 
To Some Moderate | Toa Large 
Not at All Extent Extent Extent 


% % % % 
Differentiating instruction 


Identifying skills that needs to be taught or retaught 


Monitoring student reading progress 


Creating instructional groups (in-class) 
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After-School Program 


How often do you meet with English language arts teachers at the following grade levels to 
discuss instruction-related issues regarding your work with students in the AMP after- 


school program? 


Total 


Less than 
once a 
month 


1-3 times a 
month 


1-3 times a 
week 


4-5 times a 
week 


Valid N 


Grade 6 teachers 


% 


% 


% 


% 


Grade 7 teachers 


Grade 8 teachers 


Overall (consider all the teachers that you work 
with regardless of the grade level they teach) 


How often do you meet with SIXTH-GRADE classroom teachers to discuss implementing 
each of the following instructional methods with students in the AMP after-school 


program? 


Less than 
once a 
month 


1-3 times 
a month 


1-3 times 
a week 


4-5 times 
a week 


Differentiated instruction 


% 


% 


% 


% 


Student groupings 


Use of AMP materials 


Using specific AMP and Striving Readers 
instructional techniques for comprehension 
instruction 


Using specific AMP and Striving Readers 
instructional techniques for vocabulary instruction 


Using specific AMP and Striving Readers 
instructional techniques for fluency instruction 


Discussing specific students' reading progress 


Coordinating instruction 


Using student assessment data for instructional 
planning 
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After-School Program 
Handheld Computers 


Do you use handheld computers (Palm Pilots) to teach literacy during the AMP class? 


Total 
Count 


[Pe a 


If you are not using handheld computers during the AMP class, please indicate why you 
are not using them. 


Not 
Checked Checked 


% % 
The computers have not been made available for AMP classes 100.0% 


The computers and associated software do not integrate well with 


0 
the AMP program 90.0% 


Some or all of the computers are not working properly 100.0% 


Some or all of the necessary software applications have not been 
installed on the computers 


80.0% 


I have not received sufficient professional development to feel 
comfortable using them 


100.0% 


Because the Striving Readers program provides only 10 computers 
per classroom, and I do not like to have some students use them 100.0% 
while others can not 


I do not feel that they offer sufficient added benefit compared to 
traditional media (e.g., print, paper and pencil) to be worth the 
trouble 


Other 


Other reasons for not using handheld computers during the AMP class. 


AMP software is not compatible 
with handheld computers. 

I use the AMP materials 
exclusively. 

Originally, I was taught not to stray 
from the lesson as it is written. 
Palms were not a part of the original 
plan so I was not used to working 
with them. 

Too much teacher supervision 
needed and too many groups 

used AMP materials 
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Respondent Information 


At which grade level(s) are you providing targeted intervention support for Tier 2 and 3 
students in ELA classrooms this year (2009-2010)? 


Not 
Checked Checked 


% % 

100.0% 
69.0% 
62.1% 


Not 
Checked Checked 


% % 

100.0% 
44.8% 
44.8% 


Experience 


Valid N Minimum Maximum 


How many years have you been teaching? 


How many years have you been teaching at this school? 


How many years have you been teaching reading? 


How many years have you been an LIT? 
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Appendix M: Year 4 Professional Development Schedule 


Summary of Professional Development Sessions 


Target Population 


Session 


Duration 


Principals 


Bi-Monthly sessions 


3 hrs./session X 6 sessions 


Cohort | & Il LITs 


Meetings with coordinators 


6 hrs./session X 17 sessions 


Teachers and LITs 


Teachers’ summer institute 


3.5 hrs/session X 2 sessions 
7 hrs/session X | session 


Teachers, LITs & Administrators 


Follow-up institutes 


e | hr./session X 3 sessions 
e 2hrs./session X | session 
e 4.5 hrs-6 hrs./session X 3 sessions 


Librarians 


Library Course 


3.5 hrs./session X 5 sessions 


Professional Development Activities 


Literacy Intervention Teachers (LITs) 


2009-2010 School Year 


D Duration ¢ Intended # of eligible # 
ate Topic na ae 4 
(Hours) recipients participants attending 

07/28/09 3 Overview of the project; introduction to the Striving Readers core components, All new teachers/ LITs in ) ) 
assessments, creating groups for differentiation; technology Track E schools : - 
Using data to create groups for differentiation, implementing the whole/part/whole Track E LITs with 4 

08/06/09 3.5 instructional framework, incorporating technology to increase motivation and fobhes 25 19 
engagement, planning 

08/21/09 6 Roles and Responsibilities; Data Driven Interventions Track E and New LITs 8 8 

08/28/09 6 Evaluation Process Track E and New LITs 9 9 

09/01/09 35 Overview of the project; introduction to the Striving Readers core components, All new teachers/ LITS ? ? 
assessments, creating groups for differentiation; technology 
Using data to create groups for differentiation, implementing the whole/part/whole 

09/03/09 3.5 instructional framework, incorporating technology to increase motivation and All LITs with 4 teachers ? ? 
engagement, planning 

09/11/09 6 Grant Fidelity and Implementation; AMP Training; Technology Cohorts | & Il 29 29 

09/18/09 6 Data Driven Interventions; Word Study; Technology Cohorts | & Il 22 20 

10/16/09 6 AlMSweb Progress Monitoring Training Cohorts | & Il 29 26 

10/30/09 6 AlMSweb Progress Monitoring Training Cohorts | & Il 29 27 

11/13/09 6 Collaborative Session with Teachers Cohorts | & Il 54 36 

12/11/09 6 Creating a Theory of Action; AIMSweb Training; Grand Rounds Interventions Cohorts | & Il 29 27 

ON/I5/10 6 Collaborative Session with Principals; Practices Worthy of Attention; Differentiation Cohorts | & Il 9 8 
with Technology 

02/05/10 6 Conferencing with Students; Designing and Implementing Interventions; Technology; Cohorts | & Il 9 25 
Sharing our Practices 

02/26/10 6 The Evolution of the LIT; IRA Conference Preparation; Professional Study Groups Cohorts | & Il 29 27 

03/12/10 6 Electronic Books; Palm Pilot Technology; Evolution of the Role of the LIT Cohorts | & Il 29 27 

04/23/10 6 AlMSweb Data to Design Interventions; Introduction to Rtl Cohorts | & Il 29 24 

04/30/10 6 Writing Workshop-Coaching and Supporting; Round Table Discussions/IRA Cohorts | & Il 29 25 

05/14/10 6 Data Analysis and End of Year Documents Cohorts | & Il 29 27 

05/21/10 6 Data: Deep Dive Into Assessment Data; Creating and Action Plan Based on Component Cohorts | & Il 9 7 
Implementation 

06/11/10 6 Goal Setting and Planning for 2010-2011 SY; Common Core Standards; Logic Model Cohorts | & Il 29 29 


Professional Development Activities 


All Striving Readers Treatment Schools Principals 
2009-2010 School Year 


eae # of eligible 
Date allotted to Topic hee # attending 
the session Pao 
9/17/09 3.5 hours 29 24 
11/20/09 3.5 hours Break down of High 29 19 
Fidelity Implementation 
I/15/10 3.5 hours for the remainder of the 29 22 
3/12/10 3.5 hours grant. What it means for 29 17 
the students, teachers, 
5/21/10 3.5 hours and all stakeholders. 29 22 
6/9/10 3.5 hours 29 16 
Library Professional Development Activities 
Librarians 
2009-2010 School Year 
Time oa 
Date allotted to Topic a pheligiDhe # attending 
the session aac ge 
12/10/09 3.5 hours Special Topics in School Libraries 26 16 
1/27/10 3.5 hours Special Topics in School Libraries 24 II 
2/24/10 3.5 hours Special Topics in School Libraries 24 9 
4/28/10 3.5 hours Special Topics in School Libraries 24 8 
5/26/10 3.5 hours Special Topics in School Libraries 24 8 


Summer Institute - Professional Development 


All 6-8 Grade Classroom and Resource Teachers, and Literacy Intervention Teachers 


2009-2010 School Year 


Time eo 
Date allotted to # ek oligibte ‘i : 
ye eecaion participants attending 
6/16/09 3.5 hours 19] 186 
6/17/09 3.5 hours 184 176 
6/18/09 7 hours 174 170 


Follow up Institutes - Professional Development 
All 6-8 Grade Classroom and Resource Teachers, and Literacy Intervention Teachers 
2009-2010 School Year 
Time allotted to 


Date the session Topic 
(Hours) 


# of eligible Actual number 
participants of participants 


(Multiple workshops provided 
throughout the day). 
Classroom Discourse; Data 
Analysis; Lessons with 


11/06/09 5.75 Technology; Using PALMS; 216 14] 
Literacy in the Math 
Classroom; Literacy in the 
Science Classroom 
12/10/09 2.25 Workshop for Social Studies 7 5 
Teachers 
12/10/09 2.25 Writing Workshop 38 26 
1/12/10 | Using Science and Math Text 67 10 
Sets 
1/13/10 I 67 4 
1/19/10 I Using Social Studies Text Sets 44 6 
1/20/10 2.5 Writing Workshop 44 12 
2/10/10 2 Writing Workshop 35 | 
(Multiple workshops) Word 
Study for Reading, Vocabulary 
and Thinking; Word Study 
2/20/10 5 Instruction to Integrate 215 86 


Phonics, Vocabulary and 
Spelling; Morphology at Work; 
Session for LITs and 
Collaborating Teachers 


Appendix N: Year 4 School Case Study Results 


Appendix O: Detailed Analysis Results for Overall 
Program Impact Analyses 


Hierarchical linear modeling (HLM) was used for the intent-to-treat (ITT) impact analyses of the 
overall program impact because the method takes into account the multi-level structure of the 
data (i.e., students are nested within schools) while allowing statistical control of multiple 
covariates. All HLM analyses were conducted using HLM 6.0 software. The estimation 
approach was Restricted Maximum Likelihood (REML). 


Overall Program Impact 


Cross-sectional HLM analyses were conducted to evaluate the impact of the overall program 
impact on four ITT groups: (1) Analytic Group 1 — all students at Tiers 1-3 in grades 6-8 during 
SY 2009-2010; (2) Analytic Group 2 — all students entering 6th grade at all tiers either in SY 
2008-2009 or in SY 2009-2010; (3) Analytic Group 3 — all students entering 6th grade in SY 
2008-2009; and (4) Analytic Group 4 — all students entering 6th grade in SY 2007-2008. 


For Analytic Group 1 — all students at Tiers 1-3 in grades 6-8 during SY 2009-2010 (the 
average impact of the blended intervention over different amounts of intended treatment at 
the end of the fourth project year): 
e Model 1: Average overall program impact main effect model (FULL MODEL); did not 
include interactions between treatment and subgroups of students (e.g., NCLB, grade) 
e Model 2: Average overall program impact main effect model (FINAL MODEL); did not 
include interactions between treatment and subgroups of students (e.g., NCLB, grade) 
e Model 3: Average overall program impact interaction model (FULL MODEL); included 
interactions between treatment and subgroups of students (e.g., NCLB, grade) 
e Model 4: Average overall program impact interaction model (FULL MODEL); included 
interactions between treatment and subgroups of students (e.g., NCLB, grade) 


For Analytic Group 2 — all students entering 6th grade at all tiers either in SY 2008-2009 or 
in SY 2009-2010 (one-year overall program impact): 
e Model 5: one-year overall program impact main effect model (FULL MODEL); did not 
include interactions between treatment and subgroups of students (e.g., NCLB, grade) 
e Model 6: one-year overall program impact main effect model (FINAL MODEL); did not 
include interactions between treatment and subgroups of students (e.g., NCLB, grade) 


For Analytic Group 3 — all students entering 6th grade at all tiers in SY 2008-2009 (two- 
year overall program impact): 
e Model 7: two-year overall program impact main effect model (FULL MODEL); did not 
include interactions between treatment and subgroups of students (e.g., NCLB, grade) 
e Model 8: two-year overall program impact main effect model (FINAL MODEL); did not 
include interactions between treatment and subgroups of students (e.g., NCLB, grade) 


For Analytic Group 4 — all students entering 6th grade at all tiers in SY 2007-2008 (three- 
year overall program impact): 
e Model 9: three-year overall program impact main effect model (FULL MODEL); did not 
include interactions between treatment and subgroups of students (e.g., NCLB, grade) 
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e Model 10: three-year overall program impact main effect model (FINAL MODEL); did 
not include interactions between treatment and subgroups of students (e.g., NCLB, grade) 


The specifications of the ten models, the selection and centering of covariates and the treatment 
of missing data are discussed in greater detail in this section. 


Model Specifications 


Model 1: Average overall program impact main effect model (FULL MODEL) 

Level 1: 
ISAT10j = Bo + Bi; (GENDER, — GENDER..) + Bj (IEP; — IEP..) + B3; (LEP; — LEP..)+ 
fa; (LUNCH; — LUNCH..)+ Bs; (BLACK, — BLACK..)+ Boj (HISPANIC; — HISPANIC..) + 
Br (GRD7, — GRDT..) + Bg; (GRD8; — GRD8..) + fo; (BASEISAT; — BASEISAT..)+ Bj 
(BASEMATH; — BASEMATH..) + B11; (TARGETED, — TARGETED..) + Bix (INTENSIVE; 


— INTENSIVE..)+ rj 


where 
ISAT10j represents the 2010 ISAT reading scale score for student 7 in school j 


fo; represents the mean for school 7 adjusted for the student-level covariates 


/ij— P12; represent the regression coefficients for school /, associated with the different 
student-level covariates 


r represents the random error associated with the achievement score of student i in 
school j 


Level 2: 
Boj = yoo + yor (PMIN; — PMIN.) + yo2 (SIZE; — SIZE.) + yo3 (PREAD; — PREAD.)+ yoa 
(PFEM; — PFEM.)+ yos (PSPED; — PSPED.) + yoo (PLEP; — PLEP.) + yo1 (PLUNCH; — 
PLUNCH.) + yos (COHORT; — COHORT.) + yoo (TRT;) + uj, 


By = 10 
Boj = 20 
B3j = Y30 
Bay = Ya0 
Bs = 750 
Boj = Yoo 
By = Y70 
Bsj = 80 
Boj = Yoo 
Pio; = 7100 
Bij = V110 
Br, = V120 


where 


yoo represents the average 2010 ISAT score for control schools 


401 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


¥10— Y120 represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 2: Average overall program impact main effect model (FINAL MODEL) 

Level 1: 
ISAT10j = Boj + Bi; (GENDER; — GENDER..) + Bj (IEP; — IEP..) + B3; (LEP; — LEP..)+ 
Ba; (LUNCH; — LUNCH...) + fs; (BLACK, — BLACK..)+ Bo; (HISPANIC, — HISPANIC..) + 
Bx (GRD7;; — GRD7..) + Bg (GRD8; — GRDB8..) + Bo; (BASEISAT; — BASEISAT..)+ Boj 
(BASEMATH; — BASEMATH..) + B11; (TARGETED, — TARGETED..) + Bx (INTENSIVE; 
— INTENSIVE..)+ rij 


where 


ISAT10j represents the 2010 ISAT reading scale score for student 7 in school j 
fo; represents the mean for school j adjusted for the student-level covariates 


fij— P12; represent the regression coefficients for school /, associated with the different 
student-level covariates 


rj tepresents the random error associated with the achievement score of student i in 
school j 


Level 2: 
Boj = yoo + yor (PMIN; — PMIN.) + yor (PREAD; — PREAD.) + yo3 (PSPED; — PSPED.) + yo4 
(PLEP; — PLEP.)+ yos (PLUNCH; — PLUNCH.)+ yos (COHORT; — COHORT.)+ yo 


(TRT)) + Uoj, 
By = Yi10 
Boj = Y20 
B3j = 30 
aj = Ya0 
B5j = 50 
Boj = Yoo 
By = Y70 
Bs = Ys0 
Boj = Yoo 
Proj = 7100 
Bij = V110 
Br = V120 


where 


yoo represents the average 2010 ISAT score for control schools 


401 — Yo7 represent the regression coefficients associated with the different school-level 
covariates 


¥10— Y120 represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 3: Average overall program impact interaction model (FULL MODEL) 


Level 


1: 


ISAT10j; = Bo + Bi; (GENDER; — GENDER..) + Bj (IEP; — IEP..) + 3; (LEP; — LEP..)+ 
Ba; (LUNCH; — LUNCH...) + fs; (BLACK, — BLACK..)+ Bo; (HISPANIC, — HISPANIC..) + 
Bx (GRD7;; — GRD7..) + Bg (GRD8; — GRDB8..) + Bo; (BASEISAT; — BASEISAT..)+ Boj 
(BASEMATH; — BASEMATH..) + B11; (TARGETED, — TARGETED..) + Bx (INTENSIVE; 
— INTENSIVE..)+ rij 


where 


Level 


ISAT10j represents the 2010 ISAT reading scale score for student 7 in school j 
fo; represents the mean for school j adjusted for the student-level covariates 


fij— P12; represent the regression coefficients for school /, associated with the different 
student-level covariates 


rj represents the random error associated with the achievement score of student i in 
school j 


2: 


Boj = yoo + yor (PMIN; — PMIN.)+ yo2 (SIZE; — SIZE.) + yo3 (PREAD; — PREAD.)+ yoa 
(PFEM, — PFEM.) + yos (PSPED, — PSPED.) + yos(PLEP; — PLEP.) + yo7 (PLUNCH; — 


PLUNCH.) + yos (COHORT; — COHORT.) + yoo (TRT)) + unj, 


Ay =yi0 + yu (TRT) 
Po = y20 + Yai (TRT)) 
Bs = y30 + y31 (TRT)) 
Baj = yao + yar (TRT;) 
Bs = y50 + ys1 (TRT;) 
Boj = Yoo + Yor (TRT,) 
By = 20+ yn (TRT)) 
Bs = 80 + ys1 (TRT;) 
Boj = yoo + yor (TRT;) 
Bro = Y100 * 101 (TRT;)) 
Buy = yuo + yin (TRT;) 
Pia = Yi20 + Yi21 (TRT)) 
where 


yoo represents the average 2010 ISAT score for control schools 


401 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


¥10— Y120 represent the intercepts for the regression coefficients associated with the 
different student-level covariates across schools 


¥11—Y121 represent the coefficients for the interactions between the different student-level 
covariates and school treatment 


oj represents the random error associated with school j 


Model 4: Average overall program impact interaction model (FINAL MODEL) 

Level 1: 
ISAT10j = Boj + Bi; (GENDER; — GENDER..) + Bj (IEP; — IEP..) + 3; (LEP; — LEP..)+ 
a; (LUNCH; — LUNCH...) + fs; (BLACK, — BLACK..)+ Bo; (HISPANIC, — HISPANIC..) + 
Bx (GRD7;; — GRD7..) + Bg (GRD8; — GRDB8..) + Bo; (BASEISAT; — BASEISAT..)+ Boj 
(BASEMATH; — BASEMATH..) + B11; (TARGETED, — TARGETED..) + Bx (INTENSIVE; 
— INTENSIVE..)+ rij 


where 


ISAT10j represents the 2010 ISAT reading scale score for student 7 in school j 
fo; represents the mean for school j adjusted for the student-level covariates 


fij— P12; represent the regression coefficients for school /, associated with the different 
student-level covariates 


rj represents the random error associated with the achievement score of student i in 
school j 


Level 2: 
Boj = yoo + yor (PREAD; — PREAD.) + yo2 (PSPED; — PSPED.) + yo3(PLEP; — PLEP.) + yos 
(COHORT; — COHORT.) + yos (TRT;) + uo;, 


By = Y10 
Boj = Y20 
Bs = 30 
Baj = Y40 
Bs; = 50 


Bo = Yoo + yor (TRT)) 
By = 70 + yn (IRT)) 
Bs = ys0 + ys1 (TRT;) 


Boj = Yo0 

Proj = V100 

Pai = V110 

Biz = 120 
where 


yoo represents the average 2010 ISAT score for control schools 


¥o1— Yos represent the regression coefficients associated with the different school-level 
covariates 


Y10—Yso and yoo — y120 represent the common regression coefficients associated with the 
corresponding student-level covariates for each school 


Yoo — 80 represent the intercepts for the regression coefficients associated with the 
corresponding student-level covariates across schools 


¥61— gi represent the coefficients for the interactions between the corresponding student- 
level covariates and school treatment 


uo; represents the random error associated with school j 


Model 5: one-year overall program impact main effect model (FULL MODEL) 

Level 1: 
ISAT0910; = Boj + Bij (SY — SY..) + Bx (GENDER, — GENDER..) + 3 (IEP; — IEP..) + Baj 
(LEP; — LEP..) + Bs; (LUNCH; — LUNCH..)+ Boj (BLACK, — BLACK..)+ Bx; (HISPANIC; 
— HISPANIC..) + Bg; (BASEISAT; — BASEISAT..) + Bo; (BASEMATH; — BASEMATH..) + 


Bij (TARGETED; — TARGETED..) + 1; INTENSIVE; — INTENSIVE..) + rj 


where 
ISAT0910; represents the 2009 or 2010 ISAT reading scale score for student 7 in school j 


fo represents the mean for school 7 adjusted for the student-level covariates 


Aij— fj represent the regression coefficients for school /, associated with the different 
student-level covariates 


r represents the random error associated with the achievement score of student 7 in 
school j 


Level 2: 
Boj = yoo + yo (PMIN; — PMIN.)+ yo2 (SIZE; — SIZE.) + yo3 (PREAD; — PREAD.)+ yoa 
(PFEM; — PFEM.)+ yos (PSPED; — PSPED.) + yoo (PLEP; — PLEP.) + yo7 (PLUNCH; — 
PLUNCH.) + yos (COHORT; — COHORT.) + yoo (TRT)) + unj, 


By =y10 
Boj = Y20 
Bj = Y30 
Baj = Y40 
Bs; = 50 
Boj = Yoo 
By = 70 
Bj = Ys0 
Boj = Yoo 
Bioj = Y100 
Bij = V110 


where 
yoo represents the average ISAT outcome score for control schools 


401 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


¥10— Yio represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 6: one-year overall program impact main effect model (FINAL MODEL) 

Level 1: 
ISAT0910j; = Boj + Bij (GENDER, — GENDER..) + fj (IEP; — IEP..) + B3; (LEP — LEP..) + 
a; (LUNCH; — LUNCH...) + fs; (BLACK; — BLACK..)+ Bo; (HISPANIC; — HISPANIC..) + 
1 (BASEISAT; — BASEISAT..)+ Bs; (BASEMATH;— BASEMATH..)+ Bo; (TARGETED, — 


TARGETED..) + f\9; INTENSIVE; — INTENSIVE..)+ rij 


where 
ISATO910; represents the 2009 or 2010 ISAT reading scale score for student 7 in school j 


fo; represents the mean for school 7 adjusted for the student-level covariates 


fij— Pig represent the regression coefficients for school /, associated with the different 
student-level covariates 


r represents the random error associated with the achievement score of student 7 in 


school j 
Level 2: 
Boj = yoo + yo. (PLEP; — PLEP.) + yo2 (TRT)) + Udj, 
By = 710 
Boj = Y20 
Bs = 730 
Baj = Y40 
Bs; = Y50 
Boj = Yoo 
By = 70 
Bsj = Y80 
Boj = Yoo 
Bio = 7100 
where 


yoo represents the average ISAT outcome score for control schools 


01 — Yo2 represent the regression coefficients associated with the different school-level 
covariates 


¥10— Yio0 represent the common regression coefficients associated with the different 
student-level covariates for each school 


Uo; represents the random error associated with school j 


Model 7: two-year overall program impact main effect model (FULL MODEL) 

Level 1: 
ISAT 10; = Boj + Bi; (GENDER, — GENDER..) + 2; (IEP; — IEP..) + Bx (LEP; — LEP..)+ 
a; (LUNCH; — LUNCH...) + sj (BLACK, — BLACK..)+ Bo; (HISPANIC, — HISPANIC..) + 
1 (BASEISAT; — BASEISAT..)+ Bs; (BASEMATH; — BASEMATH..) + oj (TARGETED, — 
TARGETED..) + Big; INTENSIVE, — INTENSIVE..)+ rj 


where 
ISAT10j represents the 2010 ISAT reading scale score for student 7 in school j 


fo; represents the mean for school j adjusted for the student-level covariates 


fij— Pig represent the regression coefficients for school /, associated with the different 
student-level covariates 
r tepresents the random error associated with the achievement score of student i in 
school j 
Level 2: 

Boj = yoo + yo (PMIN; — PMIN.)+ yo2 (SIZE; — SIZE.) + yo3 (PREAD; — PREAD.)+ yoa 

(PFEM; — PFEM.)+ yos (PSPED; — PSPED.) + yoo (PLEP; — PLEP.) + yo1 (PLUNCH; — 

PLUNCH.) + yos (COHORT; — COHORT.) + yoo (TRT)) + unj, 

By = Yio 

Boj = y20 

Bs = 730 

Baj = Y40 

Bs; = 50 

Boj = Yoo 

By = 70 

Bsj = Y80 

Boj = Yoo 

Boj = Y100 

where 

yoo represents the average 2010 ISAT score for control schools 
¥o1— Yoo represent the regression coefficients associated with the different school-level 
covariates 
¥10— Y100 represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 8: two-year overall program impact main effect model (FINAL MODEL) 

Level 1: 
ISAT10; = Bo; + Bij (GENDER; — GENDER..) + Bo; (IEP; — IEP...) + Bj (LEP — LEP..)+ 
Ba; (BLACK; — BLACK..)+ Bs; (HISPANIC; — HISPANIC..) + Boj (BASEISAT; — 
BASEISAT..) + 7 (BASEMATH; — BASEMATH..) + ri 


where 
ISAT10j represents the 2010 ISAT reading scale score for student 7 in school j 


fo represents the mean for school 7 adjusted for the student-level covariates 


fij— 7 represent the regression coefficients for school j, associated with the different 
student-level covariates 


r tepresents the random error associated with the achievement score of student i in 


school j 

Level 2: 

Boj = Yoo + Yo (SIZE; — SIZE.) + yo. (PLEP; — PLEP.) + yo3 (PLUNCH, — PLUNCH.) + yoa 

(TRT)) + Uoj, 

By = Y10 

Boj = 20 

£3; = Y30 

Baj = Y40 

Bsj = 50 

Boj = Yoo 

By = Y70 
where 


yoo represents the average 2010 ISAT score for control schools 


¥o1— Yoa represent the regression coefficients associated with the different school-level 
covariates 


¥10— Y70 represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 9: three-year overall program impact main effect model (FULL MODEL) 

Level 1: 
ISAT 10; = Boj + Bi; (GENDER, — GENDER..) + 2; (IEP; — IEP..) + Bx (LEP; — LEP..)+ 
Ba; (LUNCH; — LUNCH..)+ Bs; (BLACK, — BLACK..)+ Boj (HISPANIC, — HISPANIC..) + 
1 (BASEISAT; — BASEISAT..) + Bs; (BASEMATH; — BASEMATH..) + oj (TARGETED, — 
TARGETED..) + B19; INTENSIVE, — INTENSIVE..) + rij 


where 
ISAT10j represents the 2010 ISAT reading scale score for student 7 in school j 


fo; represents the mean for school j adjusted for the student-level covariates 


fij— Pig represent the regression coefficients for school /, associated with the different 
student-level covariates 


r represents the random error associated with the achievement score of student 7 in 
school j 


Level 2: 


Boj = yoo + yo (PMIN; — PMIN.) + yo2 (SIZE; — SIZE.) + yo3 (PREAD; — PREAD.)+ yoa 
(PFEM, — PFEM.)+ yos (PSPED, — PSPED.) + yos(PLEP; — PLEP.) + yo7 (PLUNCH,; — 
PLUNCH.) + yos (COHORT; — COHORT.) + yoo (TRT)) + unj, 


By = 10 
Boj = Y20 
Bs = 730 
Baj = Y40 
Bs = 50 
Boj = Yoo 
By = 70 
Bj = Ys0 
Boj = Y90 
Boj = Y100 
where 


yoo represents the average 2010 ISAT score for control schools 


¥01 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


¥10— Y1o0 represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 10: three-year overall program impact main effect model (FINAL MODEL) 

Level 1: 
ISAT10j; = Boj + Bi; (GENDER, — GENDER..) + Bj (IEP; — IEP..) + 3; (LEP; — LEP..)+ 
fa; (LUNCH; — LUNCH..)+ Bs; (BASEISAT, — BASEISAT..)+ fo; (BASEMATH, — 


BASEMATH..) + rj 
where 
ISAT10j represents the 2010 ISAT reading scale score for student 7 in school j 


fo; represents the mean for school j adjusted for the student-level covariates 


fij— Po; represent the regression coefficients for school j, associated with the different 
student-level covariates 


r represents the random error associated with the achievement score of student i in 
school j 


Level 2: 
Boj = yoo + yor (PMIN; — PMIN.) + yor (PFEM; — PFEM.)+ yo3 (PLEP; — PLEP.)+ yoa 
(PLUNCH; — PLUNCH.) + yos (TRT;) + ug; 


By =V10 
Boj = Y20 
B3j = Y30 
Ba = 40 
Bs; = 50 
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Boj = Yoo 
where 
yoo represents the average 2010 ISAT score for control schools 


401 — Yos represent the regression coefficients associated with the different school-level 
covariates 


¥10— Yoo represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Selection of Covariates 


Tables O-1, O-2, O-3, and O-4 list the covariates that were included in full models (Models 1, 3, 
5,7 and 9). All covariates, with the exception of treatment at level 2, were grand mean centered. 
For final models, covariates with p values of .200 and above were excluded from the analysis. 


Table O-1 
Variables included in Model 1 (full main effect model) 


Variable Type Abbreviation Variables 


Dependent variable ISAT10 Spring 2010 ISAT scale score 
BLACK Black (N/Y 
HISPANIC 
GRD7 
GRD8 
BASEISAT 
BASEMATH 
TARGETED 
INTENSIVE 
GENDER 
IEP 
LUNCH 
LEP 
PMIN 
PFEM 
PREAD 
PSPED 
Level 2 predictors PLEP 
PLUNCH 
SIZE 
COHORT 
TRT 


Level | predictors 


Table O-2 
Variables included in Model 3 (full interaction model) 


Variable Type Abbreviation Variables 


Dependent variable ISAT10 Spring 2010 ISAT scale score 


Level | predictors Black (N/Y) 
HISPANIC Hispanic (N/Y) 


Variable Type Abbreviation Variables 

GRD7 Grade 7 (N/Y) 
GRD8 Grade 8 (N/Y) 
BASEISAT Baseline ISAT reading scale scores 
BASEMATH Baseline ISAT math scale scores 
TARGETED Tier 2 or Tier 3 (N/Y) 
INTENSIVE Tier 3 (N/Y) 
GENDER Gender (male/female) 
ITEP Individualized education plan/special education status (N/Y) 
LUNCH Free/reduced-price lunch eligibility (N/Y) 
LEP English language learner status (N/Y) 
BLACKxTRT, 
HISPANICxTRT, 
GRD7xTRT, 
GRD8xTRT, 
BASEISATxTRT, 
ee > | Interaction between treatment and each covariate 
INTENSIVExTRT, 
GENDERxTRT, 
IEPxTRT, 
LUNCHxTRT, 
LEPxTRT 
PMIN Proportion of minority students (non- White) 
PFEM Proportion of female students 
PREAD Proportion of students at or above grade level in reading 
PSPED Proportion of special education students 

Level 2 predictors PLEP Proportion of limited English proficiency students 
PLUNCH Proportion of free/reduced-price lunch students 
SIZE School size in targeted grades 
COHORT Cohort (Cohort 1/Cohort 2) 
TRT Treatment (control/treatment) 


Table O-3 


Variables included in Model 5 (full main effect model) 


Variable Type 


Abbreviation 


Variables 


Dependent variable 


ISAT0910 


Spring 2009 or spring 2010 ISAT scale score 


Level | predictors 


BASEMATH 


INTENSIVE 


HISPANIC 
BASEISAT 


TARGETED 


GENDER 
TEP 
LUNCH 
LEP 


“Hispanic (N/Y) 


“Gender (male/female) 


“English language learner status (N/Y) 


Students entering in SY 2009-2010 or SY 2008-2009 


Level 2 predictors 


PMIN 
PFEM 
PREAD 
PSPED 
PLEP 


Proportion of minority students (non-White) 


Variable Type Abbreviation Variables 


PLUNCH 
SIZE 

COHORT 
TRT Treatment (control/treatment) 


Table O-4 
Variables included in Models 7 and 9 (full main effect models) 


Variable Type Abbreviation Variables 


Dependent variable ISAT10 Spring 2010 ISAT scale score 
BLACK Black (N/Y) 
HISPANIC 
BASEISAT 
BASEMATH 
TARGETED 
INTENSIVE 
GENDER 
IEP 
LUNCH 
LEP English language learner status (N/Y) 
PMIN Proportion of minority students (non-White) 
PFEM 
PREAD 
PSPED 
Level 2 predictors PLEP 
PLUNCH 
SIZE 
COHORT 
TRT Treatment (control/treatment) 


Level | predictors 


Treatment of Missing Data 


When conducting the HLM analyses, listwise deletion was used to remove students with missing 
data from all analytic samples. A total of 8,127 students who were at Tiers 1-3 in grades 6-8 at 
the end of SY 2009-2010 (Analytic Group 1) were included in the analyses for Models | through 
4; a total of 5,385 students who entered 6th grade at all tiers either in SY 2008-2009 or in SY 
2009-2010 (Analytic Group 2) were included in the analyses for Models 5 and 6; there were 
altogether 2,599 students who entered 6th grade at all tiers in SY 2008-2009 (Analytic Group 3) 
included in the analyses for Models 7 and 8; and a total of 2,603 students who entered 6th grade 
at all tiers in SY 2007-2008 (Analytic Group 4) were included in the analyses for Models 9 and 
10. The analyses did not include students who had no outcome data or were missing covariates. 


Table of Analysis Samples 


Table O-5 
Summary Statistics of Outcome Variable for Overall Program Impact Analysis Samples 

School Student 
Analysis Sample Group Mean SD Sample Size Sample Size 
All students at Tiers 1-3 in | _ Control | __—237.48 | 20.696 | 82 | 4053 
grades 6-8 at end of SY 0910 Treatment | 21.118 31 4074 
(Analytic Group 1, Models 1 -4) Total 20.909 63 8127 
All students entering 6" grade Control _ 20.626 32 2692 
either in SY 0809 or SY 0910 Treatment 21.694 31 2693 
(Analytic Group 2, Models 5-6) ‘Total. 21.179 63 5385 
All students entering 6" grade in Control 22.758 32 1281 
SY 0809 Treatment 22.108 31 1318 
(Analytic Group 3, Models 7-8) Total, 22.449 63 2599 
All students entering 6" grade in Control 19.254 32 1287 
SY 0708 Treatment 20.120 31 1316 
(Analytic Group 4, Models 9-10) | 19.695 63 2603 


Model 1: Average overall program impact main effect model 


Table of Analysis Model Results 


Table O-6 


(Analytic Group 1, Full Model) 


Fixed Effects Coefficient SE t p Glass’s A 
Model for INTRCPT1 (BO) 
Intercept (G00) 237.626 0.521 456.300 0.000 -- 
PMIN (G01) 10.162 7.572 1.342 0.185 0.491 
SIZE (G02) -0.002 0.003 -0.572 0.570 -0.000 
PREAD (G03) 6.929 3.776 1.835 0.072 0.335 
PFEM (G04) 1.466 7.843 0.187 0.853 0.071 
PSPED (G05) 12.164 9.342 1.302 0.199 0.588 
PLEP (G06) 13.917 5.352 2.600 0.012 0.672 
PLUNCH (G07) -14.094 10.429 -1.351 0.182 -0.681 
COHORT (G08) -1.516 0.913 -1.660 0.102 -0.073 
TRT (G09) -0.077 0.778 -0.099 0.922 -0.004 
Model for GENDER slope (B1) 
Intercept (G10) 1.023 0.278 3.679 0.000 0.049 
Model for IEP slope (B2) 
Intercept (G20) -5.226 0.666 -7.841 0.000 -0.252 
Model for LEP slope (B3) 
Intercept (G30) -6.836 0.941 -7.266 0.000 -0.330 
Model for LUNCH slope (B4) 
Intercept (G40) -2.478 0.629 -3.941 0.000 -0.120 
Model for BLACK slope (B5) 
Intercept (G50) -3.558 0.780 -4.563 0.000 -0.172 
Model for HISPANIC slope (B6) 
Intercept (G60) -2.192 0.886 -2.473 0.014 -0.106 
Model for GRD7 slope (B7) 
Intercept (G70) 4.004 0.768 5.210 0.000 0.193 
Model for GRD8 slope (B8) 
Intercept (G80) 15.826 0.614 25.786 0.000 0.765 
Model for BASEISAT slope (B9) 
Intercept (G90) 0.318 0.015 21.666 0.000 0.015 
Model for BASEMATH slope (B10) 
Intercept (G100) 0.161 0.010 16.247 0.000 0.008 
Model for TARGETED slope (B11) 
Intercept (G1 10) -7.297 0.403 -18.127 0.000 -0.353 
Model for INTENSIVE slope (B12) 
Intercept (G120) -4.022 0.482 -8.335 0.000 -0.194 
Random Effects Wariiane df Chi-Square 
(Var. Components) (p) 
Var. in school means (Up) 8.277 53 aN 
Var. within schools (R) 162.464 
From Unconditional Model 
Random Effects : Chi-Square 
(Var, Components) Variance df (p) ICC 
Var. in school means (Up) 34.786 62 ep 0.079 
(0.000) 
Var. within schools (R) 405.188 


Model 2: Average overall program impact main effect model 


Table O-7 


(Analytic Group 1, Final Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 237.636 0.509 466.583 0.000 -- 

PMIN (G01) 11.680 6.982 1.673 0.100 0.564 

PREAD (G02) 6.647 3.664 1.814 0.075 0.321 

PSPED (G03) 12.892 8.949 1.441 0.155 0.623 

PLEP (G04) 13.554 5.228 2.593 0.013 0.655 

PLUNCH (G05) -15.378 9.779 -1.573 0.121 -0.743 

COHORT (G06) -1.449 0.897 -1.617 0.111 -0.070 

TRT (G07) -0.121 0.755 -0.160 0.874 -0.006 
Model for GENDER slope (B1) 

Intercept (G10) 1.023 0.277 3.691 0.000 0.049 
Model for IEP slope (B2) 

Intercept (G20) -5.227 0.667 -7.841 0.000 -0.253 
Model for LEP slope (B3) 

Intercept (G30) -6.834 0.940 -7.268 0.000 -0.330 
Model for LUNCH slope (B4) 

Intercept (G40) -2.476 0.628 -3.942 0.000 -0.120 
Model for BLACK slope (B5) 

Intercept (G50) -3.548 0.780 -4.550 0.000 -0.171 
Model for HISPANIC slope (B6) 

Intercept (G60) -2.198 0.884 -2.485 0.013 -0.106 
Model for GRD7 slope (B7) 

Intercept (G70) 4.006 0.767 5.221 0.000 0.194 
Model for GRD8 slope (B8) 

Intercept (G80) 15.827 0.613 25.808 0.000 0.765 
Model for BASEISAT slope (B9) 

Intercept (G90) 0.318 0.015 21.651 0.000 0.015 
Model for BASEMATH slope (B10) 

Intercept (G100) 0.161 0.010 16.289 0.000 0.008 
Model for TARGETED slope (B11) 

Intercept (G110) -7.301 0.402 -18.168 0.000 -0.353 
Model for INTENSIVE slope (B12) 

Intercept (G120) -4.022 0.482 -8.337 0.000 -0.194 

Random Effects Maishee df Chi-Square 

(Var. Components) (p) 
Var. in school means (U0) 7.951 oD 000) 
Var. within schools (R) 162.464 

From Unconditional Model 
Random Effects : Chi-Square 

(Var. Components) Variance df ) ICC 

Var. in school means (Up) 34.786 62 prepa 0.079 
(0.000) 

Var. within schools (R) 405.188 


Table O-8 
Model 3: Average overall program impact interaction model 


(Analytic Group 1, Full Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 237.502 0.519 457.258 0.000 -- 

PMIN (G01) 8.238 6.923 1.190 0.240 0.398 

SIZE (G02) -0.003 0.004 -0.735 0.465 -0.000 

PREAD (G03) 7.240 3.801 1.905 0.062 0.350 

PFEM (G04) 0.019 7.697 0.003 0.998 0.001 

PSPED (G05) 13.573 9.289 1.461 0.150 0.656 

PLEP (G06) 13.045 5.572 2.341 0.023 0.630 

PLUNCH (G07) -12.725 10.252 -1.241 0.220 -0.615 

COHORT (G08) -1.565 0.921 -1.700 0.094 -0.076 

TRT (G09) 0.110 0.770 0.143 0.888 0.005 
Model for GENDER slope (B1) 

Intercept (G10) 0.927 0.374 2.476 0.014 0.045 

TRT (G11) 0.216 0.575 0.376 0.707 0.010 
Model for IEP slope (B2) 

Intercept (G20) -6.069 0.930 -6.523 0.000 -0.293 

TRT (G21) 1.830 1.314 1.393 0.164 0.088 
Model for LEP slope (B3) 

Intercept (G30) -5.642 1.024 -5.510 0.000 -0.273 

TRT (G31) -2.399 1.858 -1.291 0.197 -0.116 
Model for LUNCH slope (B4) 

Intercept (G40) -2.688 0.757 -3.551 0.001 -0.130 

TRT (G41) 0.693 1.217 0.569 0.569 0.033 
Model for BLACK slope (B5) 

Intercept (G50) -4.077 0.871 -4.683 0.000 -0.197 

TRT (G51) 0.987 1.435 0.688 0.491 0.048 
Model for HISPANIC slope (B6) 

Intercept (G60) -3.754 1.025 -3.662 0.000 -0.181 

TRT (G61) 3.031 1.470 2.062 0.039 0.146 
Model for GRD7 slope (B7) 

Intercept (G70) 6.446 0.954 6.758 0.000 0.311 

TRT (G71) -4.848 1.248 -3.885 0.000 -0.234 
Model for GRD8 slope (B8) 

Intercept (G80) 16.771 0.886 18.938 0.000 0.810 

TRT (G81) -1.868 1.164 -1.606 0.108 -0.090 
Model for BASEISAT slope (B9) 

Intercept (G90) 0.333 0.021 15.805 0.000 0.016 

TRT (G91) -0.031 0.028 -1.109 0.268 -0.002 
Model for BASEMATH slope (B10) 

Intercept (G100) 0.144 0.014 10.503 0.000 0.007 

TRT (G101) 0.036 0.020 1.823 0.068 0.002 
Model for TARGETED slope (B11) 

Intercept (G110) -7.336 0.590 -12.431 0.000 -0.354 

TRT (G111) 0.118 0.806 0.147 0.884 0.006 
Model for INTENSIVE slope (B12) 

Intercept (G120) -3.304 0.632 -5.230 0.000 -0.160 

TRT (G121) -1.466 0.938 -1.562 0.118 -0.071 

Random Effects Marinhce df Chi-Square 

(Var. Components) (p) 

Var. in school means (U0) 8.216 53 oa 
(0.000) 


Var. within schools (R) 161.407 
From Unconditional Model 
Random Effects F Chi-Square 
(ia Camanents) Variance df ) ICC 

: 729.931 
Var. in school means (Uo) 34.786 62 (0.000) 0.079 
Var. within schools (R) 405.188 

Table O-9 


Model 4: Average overall program impact interaction model 
(Analytic Group 1, Final Model) 


Fixed Effects Coefficient SE t p Glass’s A 
Model for INTRCPT1 (BO) 
Intercept (G00) 237.481 0.525 452.312 0.000 -- 
PREAD (G01) 8.878 3.005 2.954 0.005 0.429 
PSPED (G02) 18.399 9.255 1.988 0.051 0.889 
PLEP (G03) 8.311 5.153 1.613 0.112 0.402 
COHORT (G04) -1.689 0.961 -1.759 0.084 -0.082 
TRT (G05) 0.054 0.774 0.070 0.945 0.003 
Model for GENDER slope (B1) 
Intercept (G10) 1.015 0.286 3.549 0.001 0.049 
Model for IEP slope (B2) 
Intercept (G20) -5.127 0.674 -7.606 0.000 -0.248 
Model for LEP slope (B3) 
Intercept (G30) -6.798 0.956 -7.113 0.000 -0.328 
Model for LUNCH slope (B4) 
Intercept (G40) -2.447 0.617 -3.965 0.000 -0.118 
Model for BLACK slope (B5) 
Intercept (G50) -3.465 0.734 -4.718 0.000 -0.167 
Model for HISPANIC slope (B6) 
Intercept (G60) -3.466 0.949 -3.651 0.000 -0.167 
TRT (G61) 2.493 0.933 2.672 0.008 0.120 
Model for GRD7 slope (B7) 
Intercept (G70) 6.376 0.951 6.705 0.000 0.308 
TRT (G71) -4.664 1.243 -3.752 0.000 -0,.225 
Model for GRD8 slope (B8) 
Intercept (G80) 16.692 0.880 18.975 0.000 0.807 
TRT (G81) -1.730 1.165 -1.486 0.137 -0.084 
Model for BASEISAT slope (B9) 
Intercept (G90) 0.317 0.014 22.274 0.000 0.015 
Model for BASEMATH slope (B10) 
Intercept (G100) 0.161 0.010 16.136 0.000 0.008 
Model for TARGETED slope (B11) 
Intercept (G110) -7.304 0.395 -18.486 0.000 -0.353 
Model for INTENSIVE slope (B12) 
Intercept (G120) -4.023 0.478 -8.408 0.000 -0.194 
Random Effects Tacianee df Chi-Square 
(Var. Components) (p) 
Var. in school means (U0) 7.930 57 ts 000) 
Var. within schools (R) 161.450 
From Unconditional Model 
Random Effects Variance df Chi-Square ICC 


(Var. Components) (p) 
: 729.931 
Var. in school means (Up) 34.786 62 (0.000) 0.079 
Var. within schools (R) 405.188 
Table O-10 


Model 5: one-year overall program impact main effect model 


(Analytic Group 2, Full Model) 


Fixed Effects Coefficient SE t Pp Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 230.488 0.606 380.115 0.000 -- 

PMIN (G01) 3.057 11.373 0.269 0.789 0.148 

SIZE (G02) -0.005 0.005 -1.007 0.319 -0.000 

PREAD (G03) -1.716 10.105 -0.170 0.866 -0.083 

PFEM (G04) 17.763 13.706 1.296 0.201 0.861 

PSPED (G05) -7.074 12.508 -0.566 0.574 -0.343 

PLEP (G06) 13.638 7.658 1.781 0.080 0.661 

PLUNCH (G07) -14.799 17.438 -0.849 0.400 -0.717 

COHORT (G08) -0.950 1.141 -0.833 0.409 -0.046 

TRT (G09) 1.264 0.914 1.383 0.173 0.061 
Model for SY slope (B1) 

Intercept (G10) -0.531 0.648 -0.819 0.413 -0.026 
Model for GENDER slope (B2) 

Intercept (G20) 0.684 0.439 1.558 0.119 0.033 
Model for IEP slope (B3) 

Intercept (G30) -6.807 0.973 -6.993 0.000 -0.330 
Model for LEP slope (B4) 

Intercept (G40) -4.409 0.938 -4.700 0.000 -0.214 
Model for LUNCH slope (B5) 

Intercept (G50) -1.777 0.773 -2.299 0.022 -0.086 
Model for BLACK slope (B6) 

Intercept (G60) -2.114 0.882 -2.397 0.017 -0.102 
Model for HISPANIC slope (B7) 

Intercept (G70) -1.566 0.730 -2.146 0.032 -0.076 
Model for BASEISAT slope (B8) 

Intercept (G80) 0.504 0.023 21.955 0.000 0.024 
Model for BASEMATH slope (B9) 

Intercept (G90) 0.192 0.013 15.180 0.000 0.009 
Model for TARGETED slope (B10) 

Intercept (G100) -1.457 0.632 -2.305 0.021 -0.071 
Model for INTENSIVE slope (B11) 

Intercept (G110) -1.065 0.578 -1.845 0.065 -0.052 

Random Effects Wariance df Chi-Square 

(Var. Components) (p) 
Var. in school means (U0) 15.453 53 “eee 
Var. within schools (R) 152.507 

From Unconditional Model 
Random Effects 5 Chi-Square 

(Var. Components) Variance df a ICC 

Var. in school means (U0) 45.252 62 oe 0.100 
(0.000) 

Var. within schools (R) 406.720 


Table O-11 
Model 6: one-year overall program impact main effect model 
(Analytic Group 2, Final Model) 


Fixed Effects Coefficient SE t Pp Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 230.619 0.686 336.263 0.000 -- 

PLEP (G01) 13.475 6.312 2.135 0.037 0.653 

TRT (G02) 0.925 1.010 0.916 0.363 0.045 
Model for GENDER slope (B1) 

Intercept (G10) 0.703 0.438 1.606 0.108 0.034 
Model for IEP slope (B2) 

Intercept (G20) -6.773 0.971 -6.972 0.000 -0.328 
Model for LEP slope (B3) 

Intercept (G30) -4.449 0.957 -4.649 0.000 -0.216 
Model for LUNCH slope (B4) 

Intercept (G40) -1.812 0.776 -2.336 0.020 -0.088 
Model for BLACK slope (B5) 

Intercept (G50) -2.080 0.873 -2.384 0.017 -0.101 
Model for HISPANIC slope (B6) 

Intercept (G60) -1.536 0.718 -2.139 0.032 -0.074 
Model for BASEISAT slope (B7) 

Intercept (G70) 0.505 0.023 21.729 0.000 0.024 
Model for BASEMATH slope (B8) 

Intercept (G80) 0.193 0.013 15.163 0.000 0.009 
Model for TARGETED slope (B9) 

Intercept (G90) -1.458 0.634 -2.299 0.022 -0.071 
Model for INTENSIVE slope (B10) 

Intercept (G100) -1.050 0.587 -1.790 0.073 -0.051 

Random Effects Wariance df Chi-Square 

(Var. Components) (p) 
Var. in school means (U0) 14.451 60 bea 
Var. within schools (R) 152.618 

From Unconditional Model 
Random Effects : Chi-Square 

(Var. Components) Variance df ) ICC 

Var. in school means (U0) 45.252 62 ore 0.100 
(0.000) 

Var. within schools (R) 406.720 
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Model 7: two-year overall program impact main effect model 


Table O-12 


(Analytic Group 3, Full Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 237.224 0.762 311.418 0.000 -- 

PMIN (G01) 6.094 11.071 0.550 0.584 0.268 

SIZE (G02) -0.008 0.006 -1.348 0.184 -0.000 

PREAD (G03) 5.736 7.464 0.769 0.446 0.252 

PFEM (G04) 3.780 11.522 0.328 0.744 0.166 

PSPED (G05) 14.214 13.647 1.042 0.303 0.625 

PLEP (G06) 24.701 7.277 3.395 0.002 1.085 

PLUNCH (G07) -32.844 17.570 -1.869 0.067 -1.443 

COHORT (G08) 0.375 1.253 0.299 0.766 0.016 

TRT (G09) -1.524 1.130 -1.349 0.183 -0.067 
Model for GENDER slope (B1) 

Intercept (G10) 1.936 0.583 3.318 0.001 0.085 
Model for IEP slope (B2) 

Intercept (G20) -5.380 1.118 -4.810 0.000 -0.236 
Model for LEP slope (B3) 

Intercept (G30) -9.445 1.969 -4.797 0.000 -0.415 
Model for LUNCH slope (B4) 

Intercept (G40) -1.206 1.069 -1.128 0.260 -0.053 
Model for BLACK slope (B5) 

Intercept (G50) -3.908 1.577 -2.478 0.014 -0.172 
Model for HISPANIC slope (B6) 

Intercept (G60) -2.555 1.336 -1.912 0.056 -0.112 
Model for BASEISAT slope (B7) 

Intercept (G70) 0.505 0.031 16.046 0.000 0.022 
Model for BASEMATH slope (B8) 

Intercept (G80) 0.217 0.018 12.028 0.000 0.010 
Model for TARGETED slope (B9) 

Intercept (G90) -0.824 0.913 -0.903 0.367 -0.036 
Model for INTENSIVE slope (B10) 

Intercept (G100) -0.339 0.775 -0.437 0.661 -0.015 

Random Effects Mariahes df Chi-Square 

(Var. Components) (p) 
Var. in school means (U0) 16.137 53 (600 
Var. within schools (R) 184.680 

From Unconditional Model 
Random Effects : Chi-Square 

(Var. Components) Variance df ) ICC 
Var. in school means (U0) 65.385 62 (ato, 0.129 
Var. within schools (R) 443.396 
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Model 8: two-year overall program impact main effect model 


Table O-13 


(Analytic Group 3, Final Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 237.090 0.709 334.565 0.000 -- 

SIZE (G01) -0.010 0.005 -2.054 0.044 -0.000 

PLEP (G02) 24.109 7.042 3.424 0.001 1.059 

PLUNCH (G03) -40.157 12.671 -3.169 0.003 -1.765 

TRT (G04) -1.174 1.124 -1.045 0.301 -0.052 
Model for GENDER slope (B1) 

Intercept (G10) 1.938 0.587 3.300 0.001 0.085 
Model for IEP slope (B2) 

Intercept (G20) -5.386 1.127 -4.779 0.000 -0.237 
Model for LEP slope (B3) 

Intercept (G30) -9.505 1.966 -4.835 0.000 -0.418 
Model for BLACK slope (B4) 

Intercept (G40) -4.050 1.486 -2.725 0.007 -0.178 
Model for HISPANIC slope (B5) 

Intercept (G50) -2.558 1.223 -2.092 0.036 -0.112 
Model for BASEISAT slope (B6) 

Intercept (G60) 0.526 0.023 23.047 0.000 0.023 
Model for BASEMATH slope (B7) 

Intercept (G70) 0.217 0.018 11.932 0.000 0.010 

Random Effects Warinice df Chi-Square 

(Var. Components) (p) 
Var. in school means (U0) 14.854 58 000) 
Var. within schools (R) 184.607 

From Unconditional Model 
Random Effects P Chi-Square 

(Var. Components) Variance df p) ICC 
Var. in school means (U0) 65.385 62 oo 0.129 
Var. within schools (R) 443.396 
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Model 9: three-year overall program impact main effect model 


Table O-14 


(Analytic Group 4, Full Model) 


Fixed Effects Coefficient SE t Pp Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 242.723 0.540 449.441 0.000 -- 

PMIN (G01) 12.071 6.421 1.880 0.065 0.627 

SIZE (G02) -0.002 0.003 -0.647 0.520 -0.000 

PREAD (G03) 6.222 5.794 1.074 0.288 0.323 

PFEM (G04) -16.740 8.153 -2.053 0.045 -0.869 

PSPED (G05) 13.720 10.576 1.297 0.200 0.713 

PLEP (G06) 9.392 5.601 1.677 0.099 0.488 

PLUNCH (G07) -20.267 13.069 -1.551 0.127 -1.053 

COHORT (G08) -0.667 1.094 -0.610 0.544 -0.035 

TRT (G09) -0.011 0.743 -0.015 0.989 -0.001 
Model for GENDER slope (B1) 

Intercept (G10) 1.526 0.437 3.491 0.001 0.079 
Model for IEP slope (B2) 

Intercept (G20) -6.290 0.931 -6.758 0.000 -0.327 
Model for LEP slope (B3) 

Intercept (G30) -10.871 1.711 -6.353 0.000 -0.565 
Model for LUNCH slope (B4) 

Intercept (G40) -1.365 0.928 -1.472 0.141 -0.071 
Model for BLACK slope (B5) 

Intercept (G50) 0.143 1.392 0.103 0.919 0.007 
Model for HISPANIC slope (B6) 

Intercept (G60) 0.038 0.766 0.050 0.960 0.002 
Model for BASEISAT slope (B7) 

Intercept (G70) 0.416 0.019 21.735 0.000 0.022 
Model for BASEMATH slope (B8) 

Intercept (G80) 0.156 0.015 10.613 0.000 0.008 
Model for TARGETED slope (B9) 

Intercept (G90) 0.002 0.737 0.002 0.998 0.000 
Model for INTENSIVE slope (B10) 

Intercept (G100) -0.368 0.760 -0.484 0.628 -0.019 

Random Effects Mariahee df Chi-Square 

(Var. Components) (p) 
Var. in school means (U0) 6.696 53 (5000) 
Var. within schools (R) 134.330 

From Unconditional Model 
Random Effects : Chi-Square 

(Var. Components) Variance df ) ICC 
Var. in school means (U0) 46.969 62 6006) 0.121 
Var. within schools (R) 341.289 
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Model 10: three-year overall program impact main effect model 


Table O-15 


(Analytic Group 4, Final Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 242.607 0.537 452.030 0.000 -- 

PMIN (G01) 15.668 5.950 2.633 0.011 0.814 

PFEM (G02) -19.309 8.437 -2.289 0.026 -1.003 

PLEP (G03) 13.363 4.776 2.798 0.007 0.694 

PLUNCH (G04) -33.238 10.551 -3.150 0.003 -1.726 

TRT (G05) 0.268 0.739 0.363 0.718 0.014 
Model for GENDER slope (B1) 

Intercept (G10) 1.516 0.443 3.425 0.001 0.079 
Model for IEP slope (B2) 

Intercept (G20) -6.194 0.894 -6.927 0.000 -0.322 
Model for LEP slope (B3) 

Intercept (G30) -10.792 1.727 -6.249 0.000 -0.560 
Model for LUNCH slope (B4) 

Intercept (G40) -1.345 0.920 -1.462 0.144 -0.070 
Model for BASEISAT slope (B5) 

Intercept (G50) 0.423 0.015 28.776 0.000 0.022 
Model for BASEMATH slope (B6) 

Intercept (G60) 0.156 0.015 10.450 0.000 0.008 

Random Effects Varintie df Chi-Square 

(Var. Components) (p) 
Var. in school means (U0) 6.386 7: een 
Var. within schools (R) 134.212 

From Unconditional Model 
Random Effects : Chi-Square 

(Var. Components) Variance df p) ICC 
Var. in school means (U0) 46.969 62 Geo) 0.121 
Var. within schools (R) 341.289 
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Appendix P: Comparison of ITT Populations to 
Final Analysis Samples 


All Students in Tiers 1-3 in Grades 6-8 in SY 2009-2010 


Eligible Schools (n=63) 
Students (n=12,079) 


Control Schools (n=32) 
Students (n=6,103) 


Treatment Schools (n=31) 
Students (n=5,976) 


Not Analyzed Due To Ineligibility* (n=750) Not Analyzed Due To Ineligibility* (n=674) 


All Eligible ITT Students (n=5,353) All Eligible ITT Students (n=5,302) 


-- Analyzed (n=4,053 [76% of the ITT -- Analyzed (n=4,074 [77% of the ITT 
population]) population]) 

Tier 1 (n=2,255) Tier | (n=2,320) 

Tier 2 (n=1,032) Tier 2 (n=1,041) 

Tier 3 (n=766) Tier 3 (n=713) 


-- Not Analyzed Due To Missing Data (n=1,300) -- Not Analyzed Due To Missing Data (n=1,228) 
[These groups are not mutually exclusive] [These groups are not mutually exclusive] 


Missing baseline ISAT reading score (n=838) Missing baseline ISAT reading score (n=792) 
Missing baseline ISAT math score (n=639) Missing baseline ISAT math score (n=528) 


Missing outcome ISAT score (n=614) Missing outcome ISAT score (n=588) 


“Ineligibility was due to low ISAT and/or ACCESS scores. 
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One-Year of Intervention Combined Sample 


Eligible Schools (n=63) 
Students (n=6,389) 


Control Schools (n=32) 
Students (n=3,226) 


Treatment Schools (n=31) 
Students (n=3,163) 


Not Analyzed Due To Ineligibility* (n=295) Not Analyzed Due To Ineligibility* (n=268) 
-- SY 08-09: 139 -- SY 08-09: 129 
-- SY 09-10: 156 -- SY 09-10: 139 


All Eligible ITT Students (n=2,931) All Eligible ITT Students (n=2,895) 


-- Analyzed (n=2,692 [92% of the ITT -- Analyzed (n=2,693 [93% of the ITT 
population]) population]) 

Tier | (n=1,432) Tier | (n=1,431) 

Tier 2 (n=628) Tier 2 (n=660) 

Tier 3 (n=632) Tier 3 (n=602) 


-- Not Analyzed Due To Missing Data (n=239) -- Not Analyzed Due To Missing Data (n=202) 
[These groups are not mutually exclusive] [These groups are not mutually exclusive] 


Missing baseline ISAT reading score (n=225) Missing baseline ISAT reading score (n=181) 
Missing baseline ISAT math score (n=212) Missing baseline ISAT math score (n=175) 


Missing outcome ISAT score (n=20) Missing outcome ISAT score (n=20) 


“Ineligibility was due to low ISAT and/or ACCESS scores. 
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Two-Years of Intervention Sample 


Eligible Schools (n=63) 
Students (n=3,238) 


Control Schools (n=32) 
Students (n=1,632) 


Treatment Schools (n=31) 
Students (n=1,606) 


Not Analyzed Due To Ineligibility* (n=132) -- Not Analyzed Due To Ineligibility* (n=124) 


All Eligible ITT Students (n=1,500) All Eligible ITT Students (n=1,482) 


-- Analyzed (n=1,281 [85% of the ITT -- Analyzed (n=1,318 [89% of the ITT 
population]) population]) 

Tier | (n=678) Tier | (n=689) 

Tier 2 (n=319) Tier 2 (n=331) 

Tier 3 (n=284) Tier 3 (n=298) 


-- Not Analyzed Due To Missing Data (n=219) -- Not Analyzed Due To Missing Data (n=164) 
[These groups are not mutually exclusive] [These groups are not mutually exclusive] 


Missing baseline ISAT reading score (n=134) Missing baseline ISAT reading score (n=87) 
Missing baseline ISAT math score (n=128) Missing baseline ISAT math score (n=85) 


Missing ISAT 2010 score (n=108) Missing ISAT 2010 score (n=84) 


“Ineligibility was due to low ISAT and/or ACCESS scores. 
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Three-Years of Intervention Sample 


Control Schools (n=32) 
Students (n=1,764) 


All Eligible ITT Students (n=1,764)* 


-- Analyzed (n=1,287 [73% of the ITT 
population]) 

Tier | (n=616) 

Tier 2 (n=219) 

Tier 3 (n=452) 


-- Not Analyzed Due To Missing Data (n=477) 
[These groups are not mutually exclusive] 


Missing baseline ISAT reading score (n=290) 
Missing baseline ISAT math score (n=119) 
Missing ISAT 2010 score (n=247) 


Eligible Schools (n=63) 
Students (n=3,509) 


Treatment Schools (n=31) 
Students (n=1,745) 


All Eligible ITT Students (n=1,745)* 


-- Analyzed (n=1,316 [75% of the ITT 
population]) 

Tier | (n=650) 

Tier 2 (n=214) 

Tier 3 (n=452) 


-- Not Analyzed Due To Missing Data (n=429) 
[These groups are not mutually exclusive] 


Missing baseline ISAT reading score (n=305) 
Missing baseline ISAT math score (n=81) 
Missing ISAT 2010 score (n=162) 


“In Year 2, there were no ineligibility criteria for low ISAT scores or IMAGE scores. 
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Appendix O: Individual School Characteristics by 


Cohort and Treatment Group 


Table Q-1 
School Characteristics: Cohort 1 Control Schools 


Target Grades 6-8 for School Year 2009-10 


ae oval x D a ppectal ern a Amer. African oo Other/ 
N Female LEP Education | Income | Attendance Indian | Asian | American | Hispanic | White | Multiracial 

2 202 52.2 0.0 7.0 83.6 94.8 0.0 0.0 100.0 0.0 0.0 0.0 
3 172 49.7 0.0 12.9 99.4 91.4 0.0 0.0 99.4 0.6 0.0 0.0 
5 112 50.5 1.8 18.3 90.8 95.3 0.9 0.0 70.6 27.5 0.9 0.0 
12 146 50.3 0.0 11.9 96.5 91.6 0.0 0.0 97.2 2.8 0.0 0.0 
14 201 56.0 0.5 11.0 94.8 92.8 0.5 0.0 99.0 0.5 0.0 0.0 
15 92 46.2 5.5 13.2 100.0 95.8 0.0 0.0 1.1 98.9 0.0 0.0 
16 285 49.6 13.7 12.2 97.1 92.9 1.4 0.0 57.2 41.4 0.0 0.0 
19 151 47.9 7.7 15.5 97.9 95.0 1.4 0.7 19.7 71.1 7.0 0.0 
21 463 47.0 12.5 13.4 93.8 95.0 0.2 0.2 2d 88.4 9.0 0.0 
23 395 49.6 13.4 16.5 86.6 95.9 2.9 3.1 2.4 68.2 23.4 0.0 
26 School Closed 

27 238 51.9 27.9 16.3 100.0 95.8 0.0 0.0 0.0 99.6 0.4 0.0 
28 224 57.8 16.6 7.6 88.8 96.9 1.8 10.8 35.0 28.7 23.8 0.0 
29 144 64.6 0.7 16.7 81.9 96.3 0.0 0.0 97.9 2.1 0.0 0.0 
30 71 50.7 0.0 19.7 87.3 95.4 0.0 0.0 100.0 0.0 0.0 0.0 
31 161 45.5 11.0 17.2 98.6 94.9 0.0 0.0 4.8 92.4 2.8 0.0 


Table Q-2 


School Characteristics: Cohort 1 Treatment Schools 


School 


Target Grades 6-8 for School Year 2009-10 


Number lige a a & bbe lal eal ~ Amer. iia Race Other/ 
N Female | LEP | Education | Income | Attendance Indian | Asian | American | Hispanic | White | Multiracial 
4 86 47.7 0.0 18.6 88.4 96.5 0.0 0.0 33.7 64.0 2.3 0.0 
5 161 51.9 6.9 13.1 95.6 95.0 0.0 1.3 1.9 91.9 5.0 0.0 
6 369 48.1 11.5 10.4 98.9 97.3 22 0.0 7.1 90.7 0.0 0.0 
8 193 49.2 1.1 16.0 96.3 94.2 0.0 0.0 98.9 1.1 0.0 0.0 
11 606 49.6 10.0 11.0 97.8 95.3 0.2 0.0 6.5 91.5 1.8 0.0 
13 96 41.9 0.0 8.6 100.0 91.9 0.0 0.0 100.0 0.0 0.0 0.0 
16 433 47.1 9.8 15.5 89.0 96.0 1.2 1.0 2.4 79.3 16.2 0.0 
17 98 51.1 0.0 16.0 98.9 95.0 0.0 0.0 100.0 0.0 0.0 0.0 
19 159 51.3 19.2 19.2 94.9 95.7 0.0 0.6 1.3 94.2 3.8 0.0 
20 138 47.1 2.9 22.1 93.4 96.5 0.0 0.0 89.7 10.3 0.0 0.0 
22 131 50.6 8.0 12.3 91.2 92.9 0.4 0.0 De 92.0 1.9 0.0 
24 71 47.0 3.0 22.7 100.0 91.4 3.0 0.0 92.4 4.5 0.0 0.0 
27 96 46.9 13.5 20.8 90.6 96.8 0.0 0.0 14.6 83.3 2.1 0.0 
29 168 46.7 5.9 16.4 90.1 95.8 0.7 0.0 13.2 80.9 5.3 0.0 
30 83 50.0 31.3 23.8 96.3 96.1 0.0 0.0 1.3 97.5 1.3 0.0 
31 273 41.8 18.7 13.1 97.0 96.5 0.4 24.3 7.1 57.1 11.2 0.0 
Table Q-3 
School Characteristics: Cohort 2 Control Schools 
Target Grades 6-8 for School Year 2009-10 
pee Total | % | % —% Special | % Low % — ae _— 
N Female | LEP | Education | Income | Attendance Indian Asian | American | Hispanic | White | Multiracial 

1 113 53.7 0.0 12.0 98.1 92.6 0.0 0.0 90.7 9.3 0.0 0.0 
4 148 47.2 6.3 27.1 97.9 91.9 0.7 0.0 54.2 44.4 0.7 0.0 
6 132 46.5 0.0 22.8 99.2 91.6 0.0 0.8 96.9 2.4 0.0 0.0 
7 81 40.5 2.7 14.9 94.6 93.7 0.0 0.0 90.5 5.4 4.1 0.0 
8 194 49.2 0.0 15.7 94.2 94.4 0.0 0.0 99.0 1.0 0.0 0.0 
9 170 51.2 0.0 14.0 98.8 90.2 0.0 0.0 98.8 1.2 0.0 0.0 
10 133 46.6 0.0 16.0 93.9 92.9 1.5 0.0 96.2 2.3 0.0 0.0 
11 138 47.0 0.0 16.7 94.7 94.8 0.0 0.0 100.0 0.0 0.0 0.0 
13 115 52.3 0.0 14.4 98.2 92.0 0.0 0.0 100.0 0.0 0.0 0.0 
17 90 52.1 0.0 14.1 100.0 91.0 0.0 0.0 100.0 0.0 0.0 0.0 
18 186 55.9 1.7 11.7 96.6 93.3 0.0 0.0 100.0 0.0 0.0 0.0 
20 93 43.0 9.3 18.6 98.8 91.4 0.0 0.0 74.4 24.4 1.2 0.0 
22 91 51.7 6.9 14.9 98.9 95.1 0.0 0.0 77.0 20.7 2.3 0.0 
24 School Closed 

25 127 38.9 0.0 16.7 100.0 90.7 0.0 0.0 100.0 0.0 0.0 0.0 
32 155 46.6 0.0 21.4 94.7 94.1 0.0 0.0 100.0 0.0 0.0 0.0 
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Table Q-4 
School Characteristics: Cohort 2 Treatment Schools 


Target Grades 6-8 for School Year 2009-10 


mage otal 2 Ze & Bheal re tae eo Amer. iia Race Other/ 
N Female | LEP | Education | Income | Attendance ntten Asian | American | Hispanic | White | Multiracial 
1 School Closed 
2 107 46.7 0.0 14.0 100.0 94.8 0.0 0.0 100.0 0.0 0.0 0.0 
3 100 45.0 0.0 11.0 98.0 92.6 0.0 0.0 100.0 0.0 0.0 0.0 
7 77 58.4 1.3 19.5 79.2 96.0 0.0 0.0 98.7 1.3 0.0 0.0 
9 190 54.1 0.0 12.2 99.4 94.0 0.0 0.0 98.9 1.1 0.0 0.0 
10 82 45.7 0.0 17.3 97.5 95.9 0.0 0.0 96.3 2.5 1.2 0.0 
12 98 54.7 0.0 15.8 92.6 93.9 0.0 0.0 100.0 0.0 0.0 0.0 
14 137 51.2 6.5 14.6 98.4 93.1 0.0 1.6 76.4 21.1 0.8 0.0 
15 188 50.6 0.0 17.2 97.8 94.9 0.0 0.0 98.3 1.7 0.0 0.0 
18 96 53.8 0.0 16.1 98.9 89.8 0.0 0.0 97.8 2.2 0.0 0.0 
21 262 50.8 0.8 26.9 99.2 96.5 0.0 0.0 100.0 0.0 0.0 0.0 
23 80 41.6 0.0 20.8 93.5 90.4 0.0 0.0 100.0 0.0 0.0 0.0 
25 131 45.6 0.0 20.0 97.6 92.0 1.6 0.0 98.4 0.0 0.0 0.0 
26 116 45.9 0.0 21.1 100.0 92.5 0.0 0.0 100.0 0.0 0.0 0.0 
28 222 53.6 0.0 14.8 98.1 93.7 0.0 0.0 100.0 0.0 0.0 0.0 
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Appendix R: Detailed Analysis Results for Program 
Impacts on Struggling Readers 


Hierarchical linear modeling (HLM) was also used for the intent-to-treat (ITT) impact analyses 
of the program impacts on struggling readers due to its apparent methodological advantages. All 
HLM analyses were conducted using HLM 6.0 software. The estimation approach was 
Restricted Maximum Likelihood (REML). 


Targeted and Intensive Interventions 


Cross-sectional HLM analyses were also conducted to evaluate the impact of the program on 
6th-grade Tier 2 students (who received the whole-school plus targeted interventions) and 6th- 
grade Tier 3 students (who received the whole-school plus targeted plus intensive interventions). 
Program impacts on reading proficiency were investigated respectively for six ITT groups: (1) 
Analytic Group 5 — students entering 6th grade at Tier 2 either in SY 2008-2009 or in SY 2009- 
2010; (2) Analytic Group 6 — students entering 6th grade at Tier 3 either in SY 2008-2009 or in 
SY 2009-2010; (3) Analytic Group 7 — students entering 6th grade at Tier 2 in SY 2008-2009; 
(4) Analytic Group 8 — students entering 6th grade at Tier 3 in SY 2008-2009; (5) Analytic 
Group 9 — students entering 6th grade at Tier 2 in SY 2007-2008; and (6) Analytic Group 10 — 
students entering 6th grade at Tier 3 in SY 2007-2008. 


For Analytic Group 5 — students entering 6th grade at Tier 2 either in SY 2008-2009 or in 
SY 2009-2010 (one-year whole-school plus targeted interventions impact): 

e Model 11: one-year program impact on 6th-grade Tier 2 students main effect model 
(FULL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 

e Model 12: one-year program impact on 6th-grade Tier 2 students main effect model 
(FINAL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 


For Analytic Group 6 — students entering 6th grade at Tier 3 either in SY 2008-2009 or in 
SY 2009-2010 (one-year whole-school plus targeted plus intensive interventions impact): 

e Model 13: one-year program impact on 6th-grade Tier 3 students main effect model 
(FULL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 

e Model 14: one-year program impact on 6th-grade Tier 3 students main effect model 
(FINAL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 

e Model 15: one-year program impact on 6th-grade Tier 3 students interaction model 
(FULL MODEL); included interactions between treatment and NCLB subgroups of 
students 

e Model 16: one-year program impact on 6th-grade Tier 3 students interaction model 
(FINAL MODEL); included interactions between treatment and NCLB subgroups of 
students 


For Analytic Group 7 — students entering 6th grade at Tier 2 in SY 2008-2009 (two-year 
whole-school plus targeted interventions impact): 


Model 17: two-year program impact on 6th-grade Tier 2 students main effect model 
(FULL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 

Model 18: two-year program impact on 6th-grade Tier 2 students main effect model 
(FINAL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 


For Analytic Group 8 — students entering 6th grade at Tier 3 in SY 2008-2009 (two-year 
whole-school plus targeted plus intensive interventions impact): 


Model 19: two-year program impact on 6th-grade Tier 3 students main effect model 
(FULL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 

Model 20: two-year program impact on 6th-grade Tier 3 students main effect model 
(FINAL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 

Model 21: two-year program impact on 6th-grade Tier 3 students interaction model 
(FULL MODEL); included interactions between treatment and NCLB subgroups of 
students 

Model 22: two-year program impact on 6th-grade Tier 3 students interaction model 
(FINAL MODEL); included interactions between treatment and NCLB subgroups of 
students 


For Analytic Group 9 — students entering 6th grade at Tier 2 in SY 2007-2008 (three-year 
whole-school plus targeted interventions impact): 


Model 23: three-year program impact on 6th-grade Tier 2 students main effect model 
(FULL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 

Model 24: three-year program impact on 6th-grade Tier 2 students main effect model 
(FINAL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 


For Analytic Group 10 — students entering 6th grade at Tier 3 in SY 2007-2008 (three-year 
whole-school plus targeted plus intensive interventions impact): 


Model 25: three-year program impact on 6th-grade Tier 3 students main effect model 
(FULL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 

Model 26: three-year program impact on 6th-grade Tier 3 students main effect model 
(FINAL MODEL); did not include interactions between treatment and subgroups of 
students (e.g., NCLB, grade) 

Model 27: three-year program impact on 6th-grade Tier 3 students interaction model 
(FULL MODEL); included interactions between treatment and NCLB subgroups of 
students 

Model 28: three-year program impact on 6th-grade Tier 3 students interaction model 
(FINAL MODEL); included interactions between treatment and NCLB subgroups of 
students 


The specifications for all eighteen models, the selection and centering of covariates, and the 
treatment of missing data are discussed in greater detail below. 
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Model Specifications 


Model 11: one-year program impact on 6th-grade Tier 2 students main effect model (FULL 
MODEL) 


Level 1: 
ISAT0910; = Bo; + Bij (SY; — SY..) + Bx (GENDER, — GENDER..) + B3 (IEP; — IEP..) + Ba 
(LEP; — LEP..)+ Bs; LUNCH; — LUNCH..)+ Bo; (BLACK; — BLACK..)+ B71; (HISPANIC; 
— HISPANIC..) + Bs (BASEISAT; — BASEISAT..) + Bo; (BASEMATH, — BASEMATH..) + 


ri 
where 
ISAT0910; represents the 2009 or 2010 ISAT reading scale score for student 7 in school j 


fo represents the mean for school 7 adjusted for the student-level covariates 


[1;— fo; represent the regression coefficients for school /, associated with the different 
student-level covariates 


rtepresents the random error associated with the achievement score of student 7 in 
school j 


Level 2: 
Boj = yoo + Yor (PMIN; — PMIN.)+ yop (SIZE; — SIZE.) + yo3 (PREAD, — PREAD.) + yoa 


(PFEM, — PFEM.)+ yos (PSPED; — PSPED.) + yos(PLEP; — PLEP.) + yor (PLUNCH, — 
PLUNCH.) + yog (COHORT; — COHORT.) + yoo (TRT;) + uoj, 


By =Y10 
Boj = Y20 
B3 = 730 
Baj = Y40 
Bs; = Y50 
Boj = Yoo 
By = Y70 
Bsj = Y80 
Boj = Yoo 
where 


yoo represents the average ISAT outcome score for control schools 


¥o1 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


¥10— Yoo represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 12: one-year program impact on 6th-grade Tier 2 students main effect model (FINAL 
MODEL) 


Level 1: 


ISATO9 10, = Boj + Bij UEP; — IEP..) + Bo; (LEP — LEP..)+ B3; (LUNCH, — LUNCH..)+ 
fs; (BASEISAT; — BASEISAT..)+ Bs; (BASEMATH; — BASEMATH..) + rj 


where 


ISATO910j represents the 2009 or 2010 ISAT reading scale score for student 7 in school j 
fo represents the mean for school 7 adjusted for the student-level covariates 


fij— Ps; represent the regression coefficients for school j, associated with the different 
student-level covariates 


rj tepresents the random error associated with the achievement score of student i in 
school j 


Level 2: 
Bo; = yoo + Yor (PFEM; — PFEM.) + yo2 (PLEP;— PLEP.) + yos (PLUNCH; — PLUNCH.) + 
yoa (TRT;) + uoj, 
By =y10 
Boj = Y20 
Bs = 730 
Baj = Y40 
Bs; = 50 
where 


yoo represents the average ISAT outcome score for control schools 


401 — Yoa represent the regression coefficients associated with the different school-level 
covariates 


Y10— so represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 13: one-year program impact on 6th-grade Tier 3 students main effect model (FULL 

MODEL) 

Level 1: 
ISAT0910; = Bo + Bij (SY - SY..)+ By (GENDER; — GENDER..) + [3; TEP; — IEP..) + Ba 
(LEP; — LEP..)+ Bs; (LUNCH, — LUNCH..)+ Bo (BLACK, — BLACK..)+ $1; (HISPANIC; 
— HISPANIC..) + Bg; (BASEISAT; — BASEISAT..) + Bo; (BASEMATH, — BASEMATH..) + 


ij 


where 


ISATO910; represents the 2009 or 2010 ISAT reading scale score for student 7 in school j 
fo; represents the mean for school 7 adjusted for the student-level covariates 


fij— Po; represent the regression coefficients for school j, associated with the different 
student-level covariates 


rj represents the random error associated with the achievement score of student 7 in 
school j 


Level 2: 


Boj = yoo + yor (PMIN; — PMIN.)+ yo2 (SIZE; — SIZE.) + yo3 (PREAD; — PREAD.) + yoa 
(PFEM, — PFEM.)+ yos (PSPED,— PSPED.) + yoo(PLEP; — PLEP.) + yo; (PLUNCH; — 
PLUNCH.) + yos (COHORT; — COHORT.)+ yoo (TRT)) + unj, 


By = Y10 
Boj = Y20 
Bs = 730 
Baj = Y40 
Bs = 50 
Boj = Yoo 
By = 10 
Bj = Ys0 
Boj = Y90 
where 


yoo represents the average ISAT outcome score for control schools 


401 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


Y10— Yoo represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 14: one-year program impact on 6th-grade Tier 3 students main effect model (FINAL 
MODEL) 


Level 1: 
ISAT0910; = Bo; + Bij (GENDER, — GENDER..) + Bx; IEP; — IEP..) + Bx (LEP; — LEP..)+ 
fa; (BLACK, — BLACK..)+ Bs; (BASEISAT; — BASEISAT..) + Bo; (BASEMATH, — 


BASEMATH..) + rj 
where 
ISAT0910; represents the 2009 or 2010 ISAT reading scale score for student 7 in school j 


fo; represents the mean for school j adjusted for the student-level covariates 


fij— Po; represent the regression coefficients for school j, associated with the different 
student-level covariates 


r tepresents the random error associated with the achievement score of student i in 


school j 
Level 2: 
Boj = yoo + yor (PFEM; — PFEM.) + yor (IRT,) + uoj, 
By =y10 
Boj = ¥20 
Bs = 730 
Ba = y40 
Bs; = 50 
Bo = Yoo 
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where 
yoo represents the average ISAT outcome score for control schools 


01 — Yo2 represent the regression coefficients associated with the different school-level 
covariates 


¥10— Yoo represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 15: one-year program impact on 6th-grade Tier 3 students interaction model (FULL 
MODEL) 


Level 1: 
ISAT0910; = Boj + fi; (SY - SY..) + Bs; (GENDER; — GENDER..) + f3; IEP; — IEP..) + Ba; 
(LEP; — LEP..)+ Bs; LUNCH; — LUNCH..)+ Bo; (BLACK; — BLACK..)+ $7; (HISPANIC; 
— HISPANIC..) + Bg; (BASEISAT; — BASEISAT..) + oj (BASEMATH; — BASEMATH..)+ 
rij 


where 
ISATO910; represents the 2009 or 2010 ISAT reading scale score for student 7 in school j 


fo represents the mean for school 7 adjusted for the student-level covariates 


fij— Po; represent the regression coefficients for school j, associated with the different 
student-level covariates 


rj tepresents the random error associated with the achievement score of student i in 
school j 


Level 2: 
Boj = yoo + Yor (PMIN; — PMIN.)+ yop (SIZE; — SIZE.) + yo3 (PREAD, — PREAD.) + yoa 
(PFEM, — PFEM.)+ yos (PSPED,— PSPED.) + yoo(PLEP; — PLEP.) + yo; (PLUNCH; — 
PLUNCH.) + yos (COHORT; — COHORT.)+ yoo (TRT)) + uj, 
By =yiot+ yu (PRT) 
Bo = y20 + y21 (TRT;) 
23; = 30 + y31 (TRT)) 
Baj = y4o + yar (TRT)) 
Bsj = Ys0 + Ys1 (TRT)) 
Be = yoo + yo. (TRT)) 
Ba = 70 + yn (TRT)) 
Bsj = yso + si (TRT)) 
Boj = yoo + yor (TRT;) 
where 
Yoo represents the average ISAT outcome score for control schools 


401 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


¥10— Yoo represent the intercepts for the regression coefficients associated with the 
different student-level covariates across schools 


¥11—Yo1 represent the coefficients for the interactions between the different student-level 
covariates and school treatment 


uoj represents the random error associated with school j 


Model 16: one-year program impact on 6th-grade Tier 3 students interaction model (FINAL 
MODEL) 


Level 1: 
ISAT0910; = Bo; + By (GENDER, — GENDER..) + Bx; IEP; — IEP..) + Bx (LEP; — LEP..)+ 
Ba; (LUNCH; — LUNCH..)+ Bs; (BLACK; — BLACK..)+ fo; (BASEISAT; — BASEISAT..) + 


xj (BASEMATH; — BASEMATH..)+ 1; 
where 


ISATO910j represents the 2009 or 2010 ISAT reading scale score for student 7 in school j 
fo; represents the mean for school 7 adjusted for the student-level covariates 


fij— 7 represent the regression coefficients for school j, associated with the different 
student-level covariates 


represents the random error associated with the achievement score of student i in 


school j 
Level 2: 

Boj = yoo + Yor (PFEM; — PFEM.)+ yor (TRT)}) + uoy, 
Bij = Yio 

Boj = Y20 

B3j = 30 

Baj = yao + yar (TRT)) 

Bs = 950 

Boj = Yoo 

By = V70 


where 
yoo represents the average ISAT outcome score for control schools 


01 — Yo2 represent the regression coefficients associated with the different school-level 
covariates 


¥10 30 and yso— 70 represent the common regression coefficients associated with the 
corresponding student-level covariates for each school 


y4o represents the intercept for the regression coefficient associated with the 
corresponding student-level covariate across schools 


y41 represents the coefficient for the interaction between the corresponding student-level 
covariate and school treatment 


oj represents the random error associated with school j 
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Model 17: two-year program impact on 6th-grade Tier 2 students main effect model (FULL 
MODEL) 


Level 1: 
ISAT10j = Boj + Bij (GENDER; — GENDER..) + Boj (IEP; — IEP...) + B3; (LEP; — LEP..)+ 
fa; (LUNCH; — LUNCH..)+ Bs; (BLACK; — BLACK..)+ fo; (HISPANIC; — HISPANIC...) + 
xj (BASEISAT; — BASEISAT..) + Bg; (BASEMATH; — BASEMATH..)+ ri 


where 
ISAT10j represents the 2010 ISAT reading scale score for student 7 in school j 


fo represents the mean for school 7 adjusted for the student-level covariates 


/ij— Ps; represent the regression coefficients for school j, associated with the different 
student-level covariates 


r tepresents the random error associated with the achievement score of student 7 in 
school j 


Level 2: 
Boj = yoo + yor (PMIN; — PMIN.)+ yo2 (SIZE; — SIZE.) + yo3 (PREAD; — PREAD.) + yoa 
(PFEM; — PFEM.)+ yos (PSPED;— PSPED.) + yoo (PLEP; — PLEP.) + yor (PLUNCH; — 
PLUNCH.) + yog (COHORT; — COHORT.)+ yoo (TRT;) + uojs 


By = 10 
rj = Y20 
3 = Y30 
aj = Ya0 
Bs; = 50 
Boj = Yoo 
By = 
Bs = Ys0 
where 
yoo represents the average 2010 ISAT score for control schools 


401 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


¥10— Ygo represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 18: two-year program impact on 6th-grade Tier 2 students main effect model (FINAL 

MODEL) 

Level 1: 
ISAT 10; = Boj + Bi; (GENDER, — GENDER..) + 2; (IEP; — IEP..) + Bx (LEP; — LEP..)+ 
fa; (BLACK; — BLACK..)+ Bs; (HISPANIC; — HISPANIC..) + Bo; (BASEISAT; — 
BASEISAT..) + Bj (BASEMATH; — BASEMATH..) + rij 


where 


ISAT 10; represents the 2010 ISAT reading scale score for student 7 in school j 
fo represents the mean for school 7 adjusted for the student-level covariates 


fi;— 7 represent the regression coefficients for school j, associated with the different 
student-level covariates 


rj tepresents the random error associated with the achievement score of student i in 
school j 


Level 2: 
Boj = Yoo + Yor (PLUNCH; — PLUNCH.) + yo2 (TRT)) + uo, 
By =YV10 
Boj = Y20 
f3j = 730 
Baj = Y40 
Bsj = 50 
Boj = Yoo 
Bry = 770 
where 


yoo represents the average 2010 ISAT score for control schools 


401 — Yo2 represent the regression coefficients associated with the different school-level 
covariates 


Y10— 70 represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 19: two-year program impact on 6th-grade Tier 3 students main effect model (FULL 
MODEL) 


Level 1: 
ISAT 10; = Boj + Bij (GENDER; — GENDER..) + oj (IEP; — IEP...) + B3; (LEP; — LEP..)+ 
fa; (LUNCH; — LUNCH..)+ Bs; (BLACK; — BLACK..)+ fo; (HISPANIC; — HISPANIC..) + 
rj (BASEISAT; — BASEISAT..) + Bg; (BASEMATH; — BASEMATH..)+ ri 


where 


ISAT10,; represents the 2010 ISAT reading scale score for student i in school j 
fo represents the mean for school 7 adjusted for the student-level covariates 


fij— Ps; represent the regression coefficients for school j, associated with the different 
student-level covariates 


rtepresents the random error associated with the achievement score of student 7 in 
school j 


Level 2: 


Boj = yoo + yor (PMIN; — PMIN.)+ yo2 (SIZE; — SIZE.) + yo3 (PREAD; — PREAD.) + yoa 
(PFEM, — PFEM.)+ yos (PSPED,— PSPED.) + yoo(PLEP; — PLEP.) + yo; (PLUNCH; — 
PLUNCH.) + yos (COHORT; — COHORT.)+ yoo (TRT)) + unj, 


By = Y10 
Boj = Y20 
Bs = 730 
Baj = Y40 
Bs = 50 
Boj = Yoo 
By = 70 
Bj = Ys0 
where 


yoo represents the average 2010 ISAT score for control schools 


Yo1— Yoo represent the regression coefficients associated with the different school-level 
covariates 

¥10— Ygo represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 20: two-year program impact on 6th-grade Tier 3 students main effect model (FINAL 
MODEL) 


Level 1: 
ISAT 10; = Boj + Bij (GENDER; — GENDER..) + oj (IEP; — IEP...) + B3; (LEP; — LEP..)+ 
fa; (LUNCH; — LUNCH..)+ Bs; (BLACK; — BLACK..)+ fo; (BASEISAT; — BASEISAT..) + 
fj (BASEMATH; — BASEMATH..)+ rj 


where 
ISAT10; represents the 2010 ISAT reading scale score for student 7 in school j 


fo represents the mean for school 7 adjusted for the student-level covariates 


f1;— f7 represent the regression coefficients for school j, associated with the different 
student-level covariates 


rtepresents the random error associated with the achievement score of student 7 in 
school j 


Level 2: 
Boj = yoo + yor (PREAD; — PREAD.) + yor (PSPED; — PSPED.) + yo3 (COHORT, — 


By = Y10 
Boj = Y20 
Bs = Y30 
Baj = Y40 
Bs = Y50 
Boj = Yoo 
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By =n 
where 
yoo represents the average 2010 ISAT score for control schools 


401 — Yoa represent the regression coefficients associated with the different school-level 
covariates 

¥10— Y70 represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 21: two-year program impact on 6th-grade Tier 3 students interaction model (FULL 
MODEL) 


Level 1: 
ISAT10; = Boj + Bij (GENDER; — GENDER..) + Boj (IEP; — IEP..) + Bx (LEP — LEP..)+ 
fa; (LUNCH; — LUNCH..)+ Bs; (BLACK; — BLACK..)+ fo; (HISPANIC; — HISPANIC..) + 
fy; (BASEISAT; — BASEISAT..)+ Bj (BASEMATH; — BASEMATH..) + rj 


where 
ISAT 10; represents the 2010 ISAT reading scale score for student 7 in school j 


fo represents the mean for school 7 adjusted for the student-level covariates 


fij— fs; represent the regression coefficients for school j, associated with the different 
student-level covariates 


rj tepresents the random error associated with the achievement score of student i in 
school j 


Level 2: 
Boj = Yoo + Yo. (PMIN; — PMIN.)+ yop (SIZE; — SIZE.) + yo3 (PREAD,; — PREAD.) + yoa 
(PFEM, — PFEM.)+ yos (PSPED,— PSPED.) + yoo(PLEP; — PLEP.) + yo; (PLUNCH; — 
PLUNCH.) + yos (COHORT; — COHORT.)+ yoo (TRT)) + uj, 
By = 10 + yur (LRT) 
Boj = y20 + y21 (TRT)) 
£3; = y30 + y31 (TRT)) 
Ba; = y4o + ya (TRT)) 
Bsj = 50 + ys1 (TRT;) 
Bo = Yoo + Yo. (TRT;) 
By = 70 + yn (TRT;) 
Bs = yso + ysi (TRT)) 
where 
yoo represents the average 2010 ISAT score for control schools 


401 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


¥10— Ys0 represent the intercepts for the regression coefficients associated with the 
different student-level covariates across schools 
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¥11—Ygi represent the coefficients for the interactions between the different student-level 
covariates and school treatment 


uo; represents the random error associated with school j 


Model 22: two-year program impact on 6th-grade Tier 3 students interaction model (FINAL 
MODEL) 


Level 1: 
ISAT10; = Bo; + Bj (GENDER; — GENDER..) + Bo; IEP; — IEP..) + Bs; (LEP — LEP..)+ 
Ba; (LUNCH; — LUNCH..)+ fs; (BLACK; — BLACK..)+ Bo; (BASEISAT; — BASEISAT..)+ 
1 (BASEMATH, — BASEMATH..)+ rj 

where 


ISAT10j represents the 2010 ISAT reading scale score for student 7 in school / 
fo represents the mean for school 7 adjusted for the student-level covariates 


fij— 7 represent the regression coefficients for school j, associated with the different 
student-level covariates 


represents the random error associated with the achievement score of student i in 
school j 


Level 2: 
Boj = Yoo + Yol (PREAD; ca PREAD.) + Y02 (PSPED; me PSPED.) + Y03 (COHORT; _ 


COHORT.)+ yoa (TRT,) + up 


B 17 — Y10 
Boy = 920 
£3; = y30 + y31 (TRT)) 
Baj = Y40 
B5j = 50 
Boj = Yoo 
By = Y70 
where 


yoo represents the average 2010 ISAT score for control schools 


401 — Yoa represent the regression coefficients associated with the different school-level 
covariates 


¥10—Y20 and y4o0— 770 represent the common regression coefficients associated with the 
corresponding student-level covariates for each school 


39 represents the intercept for the regression coefficient associated with the 
corresponding student-level covariate across schools 


31 represents the coefficient for the interaction between the corresponding student-level 
covariate and school treatment 


Uo; represents the random error associated with school j 
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Model 23: three-year program impact on 6th-grade Tier 2 students main effect model (FULL 
MODEL)’ 


Level 1: 
ISAT10j = Boj + Bij (GENDER; — GENDER..) + fj (IEP; — IEP..) + 3; (LUNCH; — 
LUNCH..)+ $4; (BLACK; — BLACK..)+ Bs; (HISPANIC; — HISPANIC...) + Boj 
(BASEISAT; — BASEISAT..) + B1; (BASEMATH;, — BASEMATH..) + rij 


where 
ISAT10j represents the 2010 ISAT reading scale score for student 7 in school j 


fo; represents the mean for school 7 adjusted for the student-level covariates 


/ij— 7 represent the regression coefficients for school j, associated with the different 
student-level covariates 


r tepresents the random error associated with the achievement score of student 7 in 
school j 


Level 2: 
Boj = yoo + yor (PMIN; — PMIN.)+ yo2 (SIZE; — SIZE.) + yo3 (PREAD; — PREAD.) + yoa 
(PFEM; — PFEM.)+ yos (PSPED;— PSPED.) + yoo (PLEP;— PLEP.) + yor (PLUNCH; — 
PLUNCH.) + yog (COHORT; — COHORT.) + yo9 (TRT,) + ugj, 


By = 710 
By = y20 
£3; = 730 
Baj = Y40 
B5j = Y50 
Boj = Yoo 
By = Y70 
where 


yoo represents the average 2010 ISAT score for control schools 


401 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


¥10— 70 represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 24: three-year program impact on 6th-grade Tier 2 students main effect model (FINAL 
MODEL) 


Level 1: 
ISAT10; = Boj + Bi; IEP; — IEP..) + Bj (BASEISAT; — BASEISAT..)+ B3; (BASEMATH; — 
BASEMATH..) + rj 


’ LEP is not included as one of the covariates in the full model for Analytic Group 9 because all the students in the 
sample were non-ELL students. 
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where 


ISAT 10; represents the 2010 ISAT reading scale score for student 7 in school j 
fo represents the mean for school 7 adjusted for the student-level covariates 


fij;— 3; represent the regression coefficients for school j, associated with the different 
student-level covariates 


represents the random error associated with the achievement score of student i in 
school j 


Level 2: 
Boj = 00 + yo. (PSPED; — PSPED.) + yo. (PLEP; — PLEP.) + yo3 (IRT;) + uoj, 
By = yi10 
Boj = Y20 
B3j = Y30 
where 


yoo represents the average 2010 ISAT score for control schools 


401 — Yo3 represent the regression coefficients associated with the different school-level 
covariates 


Y10— 30 represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 25: three-year program impact on 6th-grade Tier 3 students main effect model (FULL 
MODEL) 


Level 1: 
ISAT 10; = Boj + Bij (GENDER; — GENDER..) + oj (IEP; — IEP...) + B3; (LEP; — LEP..)+ 
fa; (LUNCH; — LUNCH..)+ Bs; (BLACK; — BLACK..)+ Bo; (HISPANIC; — HISPANIC..) + 
fy; (BASEISAT; — BASEISAT..)+ Bs (BASEMATH; — BASEMATH..) + rj 


where 


ISAT10; represents the 2010 ISAT reading scale score for student 7 in school j 
fo; represents the mean for school 7 adjusted for the student-level covariates 


/\;— fs; represent the regression coefficients for school j, associated with the different 
student-level covariates 


rtepresents the random error associated with the achievement score of student 7 in 
school j 


Level 2: 
Boj = yoo + yor (PMIN; — PMIN.)+ yo2 (SIZE; — SIZE.) + yo3 (PREAD; — PREAD.) + yoa 
(PFEM, — PFEM.)+ yos (PSPED,;— PSPED.) + yoo(PLEP; — PLEP.) + yo; (PLUNCH; — 
PLUNCH.) + yos (COHORT; — COHORT.)+ yoo (TRT)) + unjs 


By =Y10 
Boj = Y20 
Bs = Y30 
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Ba = Yao 


Bs; = 50 

Boj = Yoo 

By = 770 

Bj = Ys0 
where 


yoo represents the average 2010 ISAT score for control schools 


¥01 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


¥10— Ygo represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 


Model 26: three-year program impact on 6th-grade Tier 3 students main effect model (FINAL 
MODEL) 


Level 1: 
ISAT10; = Boj + Bi; (GENDER; — GENDER..) + Boj (IEP; — IEP..) + Bx (LEP — LEP..)+ 
fa; (BASEISAT, — BASEISAT..)+ fs; (BASEMATH; — BASEMATH..) + rj 


where 
ISAT10,; represents the 2010 ISAT reading scale score for student i in school 7 


fo represents the mean for school 7 adjusted for the student-level covariates 


/\;— fs; represent the regression coefficients for school /, associated with the different 
student-level covariates 


represents the random error associated with the achievement score of student 7 in 


school j 
Level 2: 
Boj = Yoo + yo. (PREAD; — PREAD.) + yop (PSPED; — PSPED.) + yo3(TRT;) + Up 
By =Y10 
fa = ¥20 
Bs = 30 
Ba; = Y40 
Bs; = 50 
where 


yoo represents the average 2010 ISAT score for control schools 


401 — Yo3 represent the regression coefficients associated with the different school-level 
covariates 


¥10— Yso represent the common regression coefficients associated with the different 
student-level covariates for each school 


uo; represents the random error associated with school j 
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Model 27: three-year program impact on 6th-grade Tier 3 students interaction model (FULL 
MODEL) 


Level 1: 
ISAT10j = Boj + Bij (GENDER; — GENDER..) + Boj (IEP; — IEP...) + B3; (LEP; — LEP..)+ 
fa; (LUNCH; — LUNCH..)+ Bs; (BLACK; — BLACK..)+ fo; (HISPANIC; — HISPANIC...) + 
xj (BASEISAT; — BASEISAT..) + Bg; (BASEMATH; — BASEMATH..)+ ri 


where 


ISAT10j represents the 2010 ISAT reading scale score for student 7 in school j 
fo represents the mean for school 7 adjusted for the student-level covariates 


/ij— Ps; represent the regression coefficients for school j, associated with the different 
student-level covariates 


r represents the random error associated with the achievement score of student 7 in 
school j 


Level 2: 
Bo; = yoo + yo1 (PMIN; — PMIN.) + yo2 (SIZE; — SIZE.) + yos (PREAD; — PREAD.) + yoa 
(PFEM,— PFEM.) + yos (PSPED; — PSPED.) + yos(PLEP;— PLEP.) + yor (PLUNCH; - 
PLUNCH.) + yos (COHORT; — COHORT.)+ yoo (TRT;) + uo), 
By =Yi0 + yur (TRT,) 
Boj = Y20 + ya1 (TRT;) 
Bs; = y30 + y31 (TRT,) 
Baj = yao + ya1 (TRT;)) 
Bs; = ys0 + ys1 (TRT,) 
Bo = Yoo + yo: (TRT)) 
By =y720 + yn (IRT;) 
Bs = yso + ysi (TRT,) 


where 


yoo represents the average 2010 ISAT score for control schools 


401 — Yoo represent the regression coefficients associated with the different school-level 
covariates 


Y10— Ys0 represent the intercepts for the regression coefficients associated with the 
different student-level covariates across schools 


¥11—Ygi represent the coefficients for the interactions between the different student-level 
covariates and school treatment 


oj represents the random error associated with school j 


Model 28: three-year program impact on 6th-grade Tier 3 students interaction model (FINAL 
MODEL) 


Level 1: 
ISAT10; = Boj + Bi; (GENDER; — GENDER..) + Boj (IEP; — IEP..) + Bx (LEP — LEP..)+ 
fa; (BASEISAT; — BASEISAT..)+ fs; (BASEMATH; — BASEMATH..) + rij 


R-16 


where 


ISAT 10; represents the 2010 ISAT reading scale score for student 7 in school j 
fo represents the mean for school 7 adjusted for the student-level covariates 


fij— Ps; represent the regression coefficients for school j, associated with the different 
student-level covariates 


represents the random error associated with the achievement score of student 7 in 
school j 


Level 2: 
Boj = yoo + Yor (PREAD; — PREAD.) + yo2 (PSPED; — PSPED.) + 3 (TRT)) + Uj, 
By = yu (IRT,) 


Boj = y20 

B3; = 730 

Baj = Y40 

Bs = Y50 
where 


yoo represents the average 2010 ISAT score for control schools 


¥o1 — Yo3 represent the regression coefficients associated with the different school-level 
covariates 


20 — Ygo represent the common regression coefficients associated with the corresponding 
student-level covariates for each school 


yi1 represents the coefficient for the interaction between the corresponding student-level 
covariate and school treatment 


uo; represents the random error associated with school j 


Selection of Covariates 
Tables R-1, R-2, R-3, and R-4 list the covariates that were included in the full models (Models 


i413, 


15, 17, 19, 21, 23, 25 and 27). All covariates, with the exception of treatment at level 2, 


were grand mean centered. Final models were derived from full models based on 
interpretability, parsimony, and model fit. Generally covariates with p values of .200 and above 
were excluded from the analysis. 
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Table R-1 


Variables included in Models 11 and 13 (full main effect models) 


Variable Type 


Abbreviation 


Variables 


Dependent variable 


ISAT0910 


Spring 2009 or spring 2010 ISAT scale score 


Level | predictors 


HISPANIC 
BASEISAT 


BASEMATH 


GENDER 
TEP 
LUNCH 
LEP 


Students entering in SY 2009-2010 or SY 2008-2009 


Level 2 predictors 


PMIN 
PFEM 
PREAD 
PSPED 
PLEP 
PLUNCH 
SIZE 
COHORT 
TRT 


Treatment (control/treatment) 
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Table R-2 


Variables included in Model 15 (full interaction model) 


Variable Type 


Abbreviation 


Variables 


Dependent variable 


ISAT0910 


Spring 2009 or spring 2010 ISAT scale score 


Level 1 predictors 


SYxTRT 
BLACKxTRT, 
HISPANICXTRT, 
BASEISATXTRT, 
BASEMATHxTRT, 
GENDERxTRT, 
IEPXTRT, 
LUNCHxTRT, 
LEPxTRT 


Students entering in SY 2009-2010 or SY 2008-2009 


Hispanic (N/Y) 


Gender (male/female) 


Free/reduced-price lunch eligibility (N/Y) 


Interaction between treatment and covariates 


Level 2 predictors 


Proportion of female students 


Proportion of special education students 


School size in targeted grades 


Treatment (control/treatment) 
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Table R-3 


Variables included in Models 17, 19, 23, and 25 (full main effect models) 


Variable Type 


Abbreviation 


Variables 


Dependent variable 


ISAT10 


Spring 2010 ISAT scale score 


Level | predictors 


BLACK 
HISPANIC 


GENDER 
TEP 
LUNCH 
LEP* 


BASEISAT 
BASEMATH 


Level 2 predictors 


PMIN 
PFEM 
PREAD 
PSPED 
PLEP 
PLUNCH 
SIZE 
COHORT 
TRT 


Treatment (control/treatment) 


“LEP is not included as one of the covariates in Model 23 because all the students in Analytic Group 9 were non-ELL students. 


Table R-4 
Variables included in Models 21 and 27 (full interaction models) 
Variable Type Abbreviation Variables 
Dependent variable ISAT10 Spring 2010 ISAT scale score 


Level | predictors 


HISPANICXTRT, 
BASEISATXTRT, 
BASEMATHxTRT, 
GENDERxTRT, 
IEPXTRT, 
LUNCHxTRT, 
LEPxTRT 


Black (N/Y) 


Baseline ISAT reading scale scores 


Individualized education plan/special education status (N/Y) 


Interaction between treatment and covariates 


Level 2 predictors 


Proportion of minority students (non-White) 


Proportion of special education students 


Proportion of free/reduced-price lunch students 


Treatment (control/treatment) 
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Treatment of Missing Data 


When carrying out the HLM analyses, listwise deletion was used to remove students with 
missing data from all analytic samples. A total of 1,288 students entering 6th grade at Tier 2 
either in SY 2008-2009 or in SY 2009-2010 (Analytic Group 5) were included in the analyses for 
Models 11 and 12; there were altogether 1,234 students who entered 6th grade at Tier 3 either in 
SY 2008-2009 or in SY 2009-2010 (Analytic Group 6) included in the analyses for Models 13 
through 16; a total of 650 students who entered 6th grade at Tier 2 in SY 2008-2009 (Analytic 
Group 7) were included in the analyses for Models 17 and 18; there were altogether 582 students 
who entered 6th grade at Tier 3 in SY 2008-2009 (Analytic Group 8) included in the analyses for 
Models 19 through 22; a total of 433 student entering 6th grade at Tier 2 in SY 2007-2008 
(Analytic Group 9) were included in the analyses for Models 23 and 24; and 904 students who 
entered 6th grade at Tier 3 in SY 2007-2008 (Analytic Group 10) were included in the analyses 
for Models 25 though 28. The analyses did not include students who had no outcome data or 
were missing covariates. 


Table of Analysis Samples 


Table R-5 
Summary Statistics of Outcome Variable for Analysis Samples of Struggling Readers 
School Student 
Analysis Sample Group Mean SD | Sample Size | Sample Size 
Students entering 6" grade atTier2 |_ Control |___ 222.98 | ___—-12.386_ 32 
either in SY 0809 or in SY 0910 Treatment 224.34 13.358 31 
(Analytic Group 5, Models 11-12) Total | 223.68 | «12.906 63 
Students entering 6" grade at Tier 3 32 
either in SY 0809 or SY 0910 31 
(Analytic Group 6, Models 13-16) 63 


Students entering 6' 
SY 0809 


* grade at Tier 2 in 


(Analytic Group 7, Models 17-18) 


Total 


228.97 


Students entering 6' 
SY 0809 


* grade at Tier 3 in 


(Analytic Group 8, Models 19-22) 


Control 


Total 


215.28 


214.81 


Students entering 6' 
SY 0708 


* grade at Tier 2 in 


(Analytic Group 9, Models 23-24) 


Control 


Total 


240.80 


240.39 


Students entering 6' 
SY 0708 


(Analytic Group 10, 


* grade at Tier 3 in 


Models 25-28) 


Control 


Total 


226.40 


226.50 
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Table of Analysis Model Results 


Table R-6 
Model 11: one-year program impact on 6th-grade Tier 2 students main effect model 
(Analytic Group 5, Full Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 222.594 0.680 327.565 0.000 -- 

PMIN (G01) 9.183 13.061 0.703 0.485 0.741 

SIZE (G02) -0.003 0.005 -0.629 0.532 -0.000 

PREAD (G03) 2.182 8.566 0.255 0.800 0.176 

PFEM (G04) 20.772 15.532 1.337 0.187 1.677 

PSPED (G05) 0.722 14.798 0.049 0.962 0.058 

PLEP (G06) 21.464 10.560 2.033 0.047 1.733 

PLUNCH (G07) -34.287 22.429 -1.529 0.132 -2.768 

COHORT (G08) 0.200 1.662 0.120 0.905 0.016 

TRT (G09) 2.053 1.096 1.873 0.066 0.166 
Model for SY slope (B1) 

Intercept (G10) -0.683 0.778 -0.878 0.380 -0.055 
Model for GENDER slope (B2) 

Intercept (G20) -0.625 0.675 -0.927 0.355 -0.050 
Model for IEP slope (B3) 

Intercept (G30) -6.144 1.180 -5.205 0.000 -0.496 
Model for LEP slope (B4) 

Intercept (G40) -6.465 2.029 -3.185 0.002 -0.522 
Model for LUNCH slope (B5) 

Intercept (G50) -3.134 1.853 -1.691 0.091 -0.253 
Model for BLACK slope (B6) 

Intercept (G60) -1.252 1.573 -0.796 0.426 -0.101 
Model for HISPANIC slope (B7) 

Intercept (G70) -1.809 1.583 -1.143 0.254 -0.146 
Model for BASEISAT slope (B8) 

Intercept (G80) 0.545 0.085 6.404 0.000 0.044 
Model for BASEMATH slope (B9) 

Intercept (G90) 0.187 0.023 8.175 0.000 0.015 

Random Effects Warlance df Chi-Square 

(Var. Components) (p) 

Var. in school means (U0) 14.745 53 ee 
(0.000) 
Var. within schools (R) 125.589 
From Unconditional Model 
Random Effects : Chi-Square 

(Var. Components) Variance df p) ICC 

Var. in school means (U0) 16.205 62 Mae 0.096 
(0.000) 

Var. within schools (R) 153.145 
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Table R-7 


Model 12: one-year program impact on 6th-grade Tier 2 students main effect model 
(Analytic Group 5, Final Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 222.495 0.680 327.197 0.000 -- 

PFEM (G01) 20.595 13.462 1.530 0.131 1.663 

PLEP (G02) 15.725 7.564 2.079 0.042 1.270 

PLUNCH (G03) -32.398 16.298 -1.988 0.051 -2.616 

TRT (G04) 2.156 1.071 2.014 0.048 0.174 
Model for IEP slope (B1) 

Intercept (G10) -5.941 1.117 -5.317 0.000 -0.480 
Model for LEP slope (B2) 

Intercept (G20) -6.680 1.915 -3.488 0.001 -0.539 
Model for LUNCH slope (B3) 

Intercept (G30) -3.195 1.812 -1.763 0.078 -0.258 
Model for BASEISAT slope (B4) 

Intercept (G40) 0.547 0.084 6.477 0.000 0.044 
Model for BASEMATH slope (B5) 

Intercept (G50) 0.189 0.022 8.467 0.000 0.015 

Random Effects Nvinice df Chi-Square 

(Var. Components) (p) 
Var. in school means (U0) 12.383 58 (O60 
Var. within schools (R) 125.750 

From Unconditional Model 
Random Effects : Chi-Square 

(Var. Components) Variance df p) ICC 

Var. in school means (U0) 16.205 62 Vee: 0.096 
(0.000) 

Var. within schools (R) 153.145 
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Table R-8 


Model 13: one-year program impact on 6th-grade Tier 3 students main effect model 
(Analytic Group 6, Full Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 209.461 0.687 304.885 0.000 -- 

PMIN (G01) 1.448 9.940 0.146 0.885 0.092 

SIZE (G02) -0.001 0.005 -0.216 0.830 -0.000 

PREAD (G03) 7.556 10.570 0.715 0.478 0.481 

PFEM (G04) 33.762 15.577 2.167 0.035 2.151 

PSPED (G05) 3.039 13.160 0.231 0.818 0.194 

PLEP (G06) 1.414 6.858 0.206 0.838 0.090 

PLUNCH (G07) 11.367 19.479 0.584 0.562 0.724 

COHORT (G08) -0.554 1.615 -0.343 0.733 -0.035 

TRT (G09) 0.946 1.055 0.897 0.374 0.060 
Model for SY slope (B1) 

Intercept (G10) 0.227 0.910 0.249 0.803 0.014 
Model for GENDER slope (B2) 

Intercept (G20) 2.336 0.922 2.533 0.012 0.149 
Model for IEP slope (B3) 

Intercept (G30) -7.055 1.317 -5.357 0.000 -0.449 
Model for LEP slope (B4) 

Intercept (G40) -3.510 1.132 -3.101 0.002 -0.224 
Model for LUNCH slope (B5) 

Intercept (G50) 0.671 2.784 0.241 0.810 0.043 
Model for BLACK slope (B6) 

Intercept (G60) -3.920 1.654 -2.370 0.018 -0.250 
Model for HISPANIC slope (B7) 

Intercept (G70) -1.874 1.702 -1.101 0.272 -0.119 
Model for BASEISAT slope (B8) 

Intercept (G80) 0.613 0.053 11.637 0.000 0.039 
Model for BASEMATH slope (B9) 

Intercept (G90) 0.180 0.034 5.282 0.000 0.011 

Random Effects Wariance df Chi-Square 

(Var. Components) (p) 

Var. in school means (U0) 11.321 53 ere 
(0.000) 
Var. within schools (R) 180.385 
From Unconditional Model 
Random Effects : Chi-Square 

(Var. Components) Variance df p) ICC 

Var. in school means (U0) 14.768 62 eae 0.060 
(0.000) 

Var. within schools (R) 230.005 
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Table R-9 


Model 14: one-year program impact on 6th-grade Tier 3 students main effect model 
(Analytic Group 6, Final Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 209.377 0.710 294.965 0.000 -- 

PFEM (G01) 34.991 13.151 2.661 0.010 2.229 

TRT (G02) 1.139 1.076 1.058 0.295 0.073 
Model for GENDER slope (B1) 

Intercept (G10) 2.364 0.911 2.595 0.010 0.151 
Model for IEP slope (B2) 

Intercept (G20) -6.911 1.300 -5.317 0.000 -0.440 
Model for LEP slope (B3) 

Intercept (G30) -3.569 1.105 -3.231 0.002 -0.227 
Model for BLACK slope (B4) 

Intercept (G40) -3.007 1.040 -2.892 0.004 -0.192 
Model for BASEISAT slope (B5) 

Intercept (G50) 0.614 0.052 11.739 0.000 0.039 
Model for BASEMATH slope (B6) 

Intercept (G60) 0.182 0.034 5.376 0.000 0.012 

Random Effects Nariance df Chi-Square 

(Var. Components) (p) 
Var. in school means (U0) 9.152 60 (60) 
Var. within schools (R) 180.051 

From Unconditional Model 
Random Effects P Chi-Square 

(Var. Components) Variance df p) ICC 

Var. in school means (U0) 14.768 62 PE 0.060 
(0.000) 

Var. within schools (R) 230.005 
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Table R-10 


Model 15: one-year program impact on 6th-grade Tier 3 students interaction model 
(Analytic Group 6, Full Model) 


Fixed Effects Coefficient SE t p Glass’s A 
Model for INTRCPT1 (BO) 
Intercept (G00) 209.316 0.677 309.291 0.000 -- 
PMIN (G01) -2.806 10.205 -0.275 0.784 -0.179 
SIZE (G02) -0.002 0.006 -0.378 0.707 -0.000 
PREAD (G03) 7.788 10.486 0.743 0.461 0.496 
PFEM (G04) 33.832 15.406 2.196 0.032 2.156 
PSPED (G05) 2.690 13.347 0.202 0.841 0.171 
PLEP (G06) -0.047 7.112 -0.007 0.995 -0.003 
PLUNCH (G07) 19.411 19.394 1.001 0.322 1.237 
COHORT (G08) -0.657 1.613 -0.407 0.685 -0.042 
TRT (G09) 1.043 1.035 1.008 0.318 0.066 
Model for SY slope (B1) 
Intercept (G10) 0.554 1.256 0.441 0.659 0.035 
TRT (G11) -0.382 1.810 -0.211 0.833 -0.024 
Model for GENDER slope (B2) 
Intercept (G20) 3.357 1.241 2.706 0.007 0.214 
TRT (G21) -2.098 1.819 -1.153 0.249 -0.134 
Model for IEP slope (B3) 
Intercept (G30) -6.280 1.878 -3.343 0.001 -0.400 
TRT (G31) -1.552 2.516 -0.617 0.537 -0.099 
Model for LEP slope (B4) 
Intercept (G40) -3.335 2.112 -1.579 0.114 -0.212 
TRT (G41) -0.522 2.429 -0.215 0.830 -0.033 
Model for LUNCH slope (B5) 
Intercept (G50) 6.579 3.395 1.938 0.052 0.419 
TRT (G51) -13.897 5.113 -2.718 0.007 -0.885 
Model for BLACK slope (B6) 
Intercept (G60) -3.564 1.774 -2.009 0.044 -0.227 
TRT (G61) -0.974 3.120 -0.312 0.755 -0.062 
Model for HISPANIC slope (B7) 
Intercept (G70) -3.209 1.843 -1.741 0.081 -0.204 
TRT (G71) 2.458 3.281 0.749 0.454 0.157 
Model for BASEISAT slope (B8) 
Intercept (G80) 0.628 0.077 8.144 0.000 0.040 
TRT (G81) -0.014 0.106 -0.128 0.899 -0.001 
Model for BASEMATH slope (B9) 
Intercept (G90) 0.214 0.037 5.788 0.000 0.014 
TRT (G91) -0.071 0.070 -1.014 0.311 -0.005 
Random Effects Watinnce df Chi-Square 
(Var. Components) (p) 
Var. in school means (U0) 11.566 53 00) 
Var. within schools (R) 179.953 
From Unconditional Model 
Random Effects : Chi-Square 
(Var. Components) Variance df p) ICC 
Var. in school means (U0) | 14.768 62 pea oges 0.060 
(0.000) 
Var. within schools (R) 230.005 
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Table R-11 


Model 16: one-year program impact on 6th-grade Tier 3 students interaction model 
(Analytic Group 6, Final Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 209.405 0.709 295.531 0.000 -- 

PFEM (G01) 35.553 13.198 2.694 0.010 2.265 

TRT (G02) 1.105 1.071 1.031 0.307 0.070 
Model for GENDER slope (B1) 

Intercept (G10) 2.401 0.923 2.602 0.010 0.153 
Model for IEP slope (B2) 

Intercept (G20) -7.036 1.290 -5.456 0.000 -0.448 
Model for LEP slope (B3) 

Intercept (G30) -3.643 1.127 -3.232 0.002 -0.232 
Model for LUNCH slope (B4) 

Intercept (G40) 5.982 3.462 1.728 0.084 0.381 

TRT (G41) -12.989 4.818 -2.696 0.008 -0.828 
Model for BLACK slope (B5) 

Intercept (G50) -3.081 1.035 -2.977 0.003 -0.196 
Model for BASEISAT slope (B6) 

Intercept (G60) 0.615 0.052 11.726 0.000 0.039 
Model for BASEMATH slope (B7) 

Intercept (G70) 0.181 0.034 5.315 0.000 0.012 

Random Effects Maniatis df Chi-Square 

(Var. Components) (p) 
Var. in school means (U0) 9.013 60 O80) 
Var. within schools (R) 179.495 

From Unconditional Model 
Random Effects 6 Chi-Square 

(Var. Components) Variance df p) ICC 

Var. in school means (U0) 14.768 62 pases 0.060 
(0.000) 

Var. within schools (R) 230.005 
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Table R-12 


Model 17: two-year program impact on 6th-grade Tier 2 students main effect model 
(Analytic Group 7, Full Model) 


Fixed Effects Coefficient SE t Pp Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 230.403 1.032 223.280 0.000 -- 

PMIN (G01) 7TASI 15.294 0.468 0.642 0.492 

SIZE (G02) -0.010 0.008 -1.252 0.216 -0.001 

PREAD (G03) -0.367 7.727 -0.047 0.963 -0.025 

PFEM (G04) -6.200 18.290 -0.339 0.736 -0.426 

PSPED (G05) 9.390 18.000 0.522 0.604 0.645 

PLEP (G06) 17.017 11.979 1.421 0.161 1.170 

PLUNCH (G07) -48.485 21.806 -2.223 0.030 -3.333 

COHORT (G08) 2.526 1.950 1.295 0.201 0.174 

TRT (G09) -1.491 1.504 -0.991 0.327 -0.102 
Model for GENDER slope (B1) 

Intercept (G10) 1.795 1.034 1.736 0.083 0.123 
Model for IEP slope (B2) 

Intercept (G20) -3.358 1.744 -1.925 0.054 -0.231 
Model for LEP slope (B3) 

Intercept (G30) -10.011 2.584 -3.874 0.000 -0.688 
Model for LUNCH slope (B4) 

Intercept (G40) 0.592 2.719 0.218 0.828 0.041 
Model for BLACK slope (B5) 

Intercept (G50) -6.540 2.745 -2.382 0.018 -0.450 
Model for HISPANIC slope (B6) 

Intercept (G60) -3.425 2.180 -1.571 0.116 -0.235 
Model for BASEISAT slope (B7) 

Intercept (G70) 0.574 0.131 4.378 0.000 0.039 
Model for BASEMATH slope (B8) 

Intercept (G80) 0.240 0.036 6.640 0.000 0.017 

Random Effects Variance df Chi-Square 

(Var. Components) (p) 

Var. in school means (U0) 18.371 53 ae 
(0.000) 
Var. within schools (R) 163.716 
From Unconditional Model 
Random Effects 6 Chi-Square 

(Var. Components) Variance df (p) ICC 
Var. in school means (U0) 15.009 62 a 0.070 
Var. within schools (R) 198.473 
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Table R-13 


Model 18: two-year program impact on 6th-grade Tier 2 students main effect model 
(Analytic Group 7, Final Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 229.852 0.963 238.616 0.000 -- 

PLUNCH (G01) -34.898 18.958 -1.841 0.070 -2.399 

TRT (G02) -1.297 1.460 -0.888 0.378 -0.089 
Model for GENDER slope (B1) 

Intercept (G10) 1.646 1.035 1.589 0.112 0.113 
Model for IEP slope (B2) 

Intercept (G20) -3.599 1.762 -2.043 0.041 -0.247 
Model for LEP slope (B3) 

Intercept (G30) -9.806 2.586 -3.792 0.000 -0.674 
Model for BLACK slope (B4) 

Intercept (G40) -5.335 2.470 -2.160 0.031 -0.367 
Model for HISPANIC slope (B5) 

Intercept (G50) -3,342 2.189 -1.527 0.127 -0.230 
Model for BASEISAT slope (B6) 

Intercept (G60) 0.588 0.131 4.496 0.000 0.040 
Model for BASEMATH slope (B7) 

Intercept (G70) 0.235 0.036 6.492 0.000 0.016 

Random Effects Wa BHCe df Chi-Square 

(Var. Components) (p) 
Var. in school means (U0) 16.234 60 Pires 
Var. within schools (R) 163.955 

From Unconditional Model 
Random Effects : Chi-Square 

(Var. Components) Variance df p) ICC 
Var. in school means (U0) 15.009 62 Tid 0.070 
Var. within schools (R) 198.473 
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Table R-14 
Model 19: two-year program impact on 6th-grade Tier 3 students main effect model 
(Analytic Group 8, Full Model) 


Fixed Effects Coefficient SE t p Glass’s A 
Model for INTRCPT1 (BO) 
Intercept (G00) 216.086 1.174 183.983 0.000 -- 
PMIN (G01) 6.320 16.350 0.387 0.700 0.372 
SIZE (G02) -0.006 0.006 -0.998 0.323 -0.000 
PREAD (G03) 21.139 8.220 2.572 0.013 1.243 
PFEM (G04) 18.066 15.759 1.146 0.257 1.062 
PSPED (G05) 41.151 21.757 1.891 0.064 2.420 
PLEP (G06) 11.057 14.814 0.746 0.459 0.650 
PLUNCH (G07) 8.955 23.502 0.381 0.704 0.527 
COHORT (G08) -2.776 1.840 -1.509 0.137 -0.163 
TRT (G09) -1.560 1.655 -0.943 0.350 -0.092 
Model for GENDER slope (B1) 
Intercept (G10) 1.989 1.242 1.601 0.110 0.117 
Model for IEP slope (B2) 
Intercept (G20) -7.491 1.655 -4.525 0.000 -0.440 
Model for LEP slope (B3) 
Intercept (G30) -11.355 2.758 -4.117 0.000 -0.668 
Model for LUNCH slope (B4) 
Intercept (G40) -11.777 4.208 -2.799 0.006 -0.692 
Model for BLACK slope (B5) 
Intercept (G50) -5.863 3.537 -1.658 0.098 -0.345 
Model for HISPANIC slope (B6) 
Intercept (G60) -1.736 3.664 -0.474 0.635 -0.102 
Model for BASEISAT slope (B7) 
Intercept (G70) 0.454 0.080 5.687 0.000 0.027 
Model for BASEMATH slope (B8) 
Intercept (G80) 0.202 0.052 3.866 0.000 0.012 
Random Effects Variance df Chi-Square 
(Var. Components) (p) 
Var. in school means (U0) 14.909 53 and 
(0.003) 
Var. within schools (R) 209.943 
From Unconditional Model 
Random Effects ; Chi-Square 
(Var. Components) Variance df (p) ICC 
Var. in school means (U0) 21.844 62 esi 0.075 
, ; (0.000) 
Var. within schools (R) 269.294 
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Table R-15 


Model 20: two-year program impact on 6th-grade Tier 3 students main effect model 
(Analytic Group 8, Final Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 216.180 1.156 187.069 0.000 -- 

PREAD (G01) 19.148 6.632 2.887 0.006 1.126 

PSPED (G02) 41.443 20.437 2.028 0.047 2.437 

COHORT (G03) -2.690 1.806 -1.489 0.142 -0.158 

TRT (G04) -1.920 1.561 -1.230 0.224 -0.113 
Model for GENDER slope (B1) 

Intercept (G10) 2.028 1.238 1.639 0.101 0.119 
Model for IEP slope (B2) 

Intercept (G20) -7.389 1.660 -4.451 0.000 -0.434 
Model for LEP slope (B3) 

Intercept (G30) -11.030 2.781 -3.966 0.000 -0.649 
Model for LUNCH slope (B4) 

Intercept (G40) -12.321 4.288 -2.873 0.005 -0.724 
Model for BLACK slope (B5) 

Intercept (G50) -4.241 1.648 -2.574 0.011 -0.249 
Model for BASEISAT slope (B6) 

Intercept (G60) 0.453 0.080 5.666 0.000 0.027 
Model for BASEMATH slope (B7) 

Intercept (G70) 0.204 0.052 3.912 0.000 0.012 

Random Effects Waris df Chi-Square 

(Var. Components) (p) 

Var. in school means (U0) 13.366 58 aan 
(0.004) 
Var. within schools (R) 209.313 
From Unconditional Model 
Random Effects : Chi-Square 

(Var. Components) Variance df p) ICC 

Var. in school means (U0) 21.844 62 pated 0.075 
(0.000) 

Var. within schools (R) 269.294 
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Table R-16 


Model 21: two-year program impact on 6th-grade Tier 3 students interaction model 
(Analytic Group 8, Full Model) 


Fixed Effects Coefficient SE t p Glass’s A 
Model for INTRCPT1 (BO) 
Intercept (G00) 216.049 1.103 195.856 0.000 -- 
PMIN (G01) 3.755 18.015 0.208 0.836 0.221 
SIZE (G02) -0.006 0.006 -0.949 0.347 -0.000 
PREAD (G03) 21.503 8.120 2.648 0.011 1.264 
PFEM (G04) 17.391 15.870 1.096 0.279 1.023 
PSPED (G05) 39.981 22.483 1.778 0.081 2.351 
PLEP (G06) 7.871 13.992 0.563 0.576 0.463 
PLUNCH (G07) 8.786 23.027 0.382 0.704 0.517 
COHORT (G08) -2.852 1.758 -1.622 0.110 -0.168 
TRT (G09) -1.638 1.561 -1.050 0.299 -0.096 
Model for GENDER slope (B1) 
Intercept (G10) 2.691 1.939 1.388 0.166 0.158 
TRT (G11) -1.147 22531 | -0.445 0.656 -0.067 
Model for IEP slope (B2) 
Intercept (G20) -9.110 2.053 -4.438 0.000 -0.536 
TRT (G21) 2.889 3.101 0.932 0.352 0.170 
Model for LEP slope (B3) 
Intercept (G30) -6.342 1.957 -3.241 0.002 -0.373 
TRT (G31) -8.929 4.239 -2.107 0.035 -0.525 
Model for LUNCH slope (B4) 
Intercept (G40) -10.667 5.147 -2.072 0.038 -0.627 
TRT (G41) -1.918 8.968 -0.214 0.831 -0.113 
Model for BLACK slope (B5) 
Intercept (G50) -4.751 3.458 -1.374 0.170 -0.279 
TRT (G51) -1.461 6.843 -0.214 0.831 -0.086 
Model for HISPANIC slope (B6) 
Intercept (G60) -1.227 4.499 -0.273 0.785 -0.072 
TRT (G61) -0.428 7.123 -0.060 0.953 -0.025 
Model for BASEISAT slope (B7) 
Intercept (G70) 0.419 0.124 3.393 0.001 0.025 
TRT (G71) 0.074 0.163 0.456 0.648 0.004 
Model for BASEMATH slope (B8) 
Intercept (G80) 0.192 0.079 2.425 0.016 0.011 
TRT (G81) 0.009 0.102 0.087 0.931 0.001 
Random Effects Ariane df Chi-Square 
(Var. Components) (p) 
Var. in school means (U0) 13.882 53 ioe 
Var. within schools (R) 211.895 
From Unconditional Model 
Random Effects : Chi-Square 
(Var. Components) Variance df p) ICC 
Var. in school means (U0) 21.844 62 sie 0.075 
(0.000) 
Var. within schools (R) 269.294 


R-32 


Table R-17 


Model 22: two-year program impact on 6th-grade Tier 3 students interaction model 
(Analytic Group 8, Final Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 216.212 1.116 193.715 0.000 -- 

PREAD (G01) 19.790 6.623 2.988 0.005 1.164 

PSPED (G02) 38.373 20.334 1.887 0.064 2.256 

COHORT (G03) -2.642 1.780 -1.484 0.143 -0.155 

TRT (G04) -1.976 1.513 -1.306 0.197 -0.116 
Model for GENDER slope (B1) 

Intercept (G10) 2.087 1.248 1.671 0.095 0.123 
Model for IEP slope (B2) 

Intercept (G20) -7.326 1.638 -4.473 0.000 -0.431 
Model for LEP slope (B3) 

Intercept (G30) -6.276 2.295 -2.734 0.007 -0.369 

TRT (G31) -8.637 4.442 -1.944 0.052 -0.508 
Model for LUNCH slope (B4) 

Intercept (G40) -12.410 4.295 -2.890 0.004 -0.730 
Model for BLACK slope (B5) 

Intercept (G50) -4.045 1.673 -2.418 0.016 -0.238 
Model for BASEISAT slope (B6) 

Intercept (G60) 0.448 0.080 5.605 0.000 0.026 
Model for BASEMATH slope (B7) 

Intercept (G70) 0.202 0.051 3.927 0.000 0.012 

Random Effects Norianee df Chi-Square 

(Var. Components) (p) 

Var. in school means (U0) 11.817 58 cet 
(0.009) 
Var. within schools (R) 209.328 
From Unconditional Model 
Random Effects é Chi-Square 

(Var. Components) Variance df p) ICC 
Var. in school means (U0) 21.844 62 pide 0.075 

, ; (0.000) 
Var. within schools (R) 269.294 
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Table R-18 
Model 23: three-year program impact on 6th-grade Tier 2 students main effect model 
(Analytic Group 9, Full Model) 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 241.245 0.961 251.153 0.000 -- 

PMIN (G01) 16.746 11.113 1.507 0.138 1.396 

SIZE (G02) 0.004 0.006 0.762 0.449 0.000 

PREAD (G03) 8.289 7.566 1.096 0.279 0.691 

PFEM (G04) -27.800 11.859 -2.344 0.023 -2.317 

PSPED (G05) 33.658 13.624 2.470 0.017 2.805 

PLEP (G06) 16.829 9.527 1.766 0.083 1.403 

PLUNCH (G07) -26.788 20.683 -1.295 0.201 -2.233 

COHORT (G08) -0.459 1.798 -0.255 0.799 -0.038 

TRT (G09) -1.165 1.321 -0.882 0.382 -0.097 
Model for GENDER slope (B1) 

Intercept (G10) 1.071 0.835 1.283 0.201 0.089 
Model for IEP slope (B2) 

Intercept (G20) -6.203 2.328 -2.665 0.008 -0.517 
Model for LUNCH slope (B3) 

Intercept (G30) 1.098 2.259 0.486 0.627 0.092 
Model for BLACK slope (B4) 

Intercept (G40) -2.633 2.483 -1.060 0.290 -0.219 
Model for HISPANIC slope (B5) 

Intercept (G50) -3.727 2.495 -1.494 0.136 -0.311 
Model for BASEISAT slope (B6) 

Intercept (G60) 0.502 0.075 6.710 0.000 0.042 
Model for BASEMATH slope (B7) 

Intercept (G70) 0.177 0.029 6.071 0.000 0.015 

Random Effects Vavinnce df Chi-Square 

(Var. Components) (p) 

Var. in school means (U0) 12.503 53 pee 
(0.000) 
Var. within schools (R) 92.752 
From Unconditional Model 
Random Effects ; Chi-Square 

(Var. Components) Variance df p) ICC 

Var. in school means (U0) 18.974 62 kee 0.126 
(0.000) 

Var. within schools (R) 132.179 
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Table R-19 


(Analytic Group 9, Final Model) 


Model 24: three-year program impact on 6th-grade Tier 2 students main effect model 


Fixed Effects Coefficient SE t Pp Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 241.048 0.936 257.660 0.000 -- 

PSPED (G01) 32.011 13.743 2.329 0.023 2.668 

PLEP (G02) 19.300 7.799 2.475 0.016 1.609 

TRT (G03) -0.771 1.266 -0.609 0.544 -0.064 
Model for IEP slope (B1) 

Intercept (G10) -5.669 2.279 -2.487 0.013 -0.473 
Model for BASEISAT slope (B2) 

Intercept (G20) 0.515 0.076 6.817 0.000 0.043 
Model for BASEMATH slope (B3) 

Intercept (G30) 0.180 0.029 6.189 0.000 0.015 

Random Effects Nacahee df Chi-Square 

(Var. Components) (p) 

Var. in school means (U0) 12.906 59 Maes 
(0.000) 
Var. within schools (R) 92.911 
From Unconditional Model 
Random Effects : Chi-Square 

(Var. Components) Variance df @) ICC 

Var. in school means (U0) 18.974 62 anal 0.126 
(0.000) 

Var. within schools (R) 132.179 
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Table R-20 


Model 25: three-year program impact on 6th-grade Tier 3 students main effect model 
(Analytic Group 10, Full Model) 
Fixed Effects Coefficient SE t p Glass’s A 
Model for INTRCPT1 (BO) 
Intercept (G00) 226.836 0.708 320.522 0.000 -- 
PMIN (G01) 14.208 13.250 1.072 0.289 0.875 
SIZE (G02) 0.000 0.004 0.102 0.920 0.000 
PREAD (G03) 12.685 6.153 2.062 0.044 0.781 
PFEM (G04) -0.434 10.481 -0.041 0.967 -0.027 
PSPED (G05) 20.372 11.300 1.803 0.077 1.254 
PLEP (G06) 10.868 12.328 0.882 0.382 0.669 
PLUNCH (G07) -1.532 16.997 -0.090 0.929 -0.094 
COHORT (G08) -1.750 1.594 -1.098 0.278 -0.108 
TRT (G09) 0.065 1.067 0.061 0.952 0.004 
Model for GENDER slope (B1) 
Intercept (G10) 2.473 0.713 3.467 0.001 0.152 
Model for IEP slope (B2) 
Intercept (G20) -5.451 1.498 -3.640 0.001 -0.336 
Model for LEP slope (B3) 
Intercept (G30) -10.962 1.560 -7.029 0.000 -0.675 
Model for LUNCH slope (B4) 
Intercept (G40) -3.079 2.945 -1.046 0.297 -0.190 
Model for BLACK slope (B5) 
Intercept (G50) 0.950 3.228 0.294 0.768 0.059 
Model for HISPANIC slope (B6) 
Intercept (G60) -1.614 2.459 -0.656 0.512 -0.099 
Model for BASEISAT slope (B7) 
Intercept (G70) 0.464 0.043 10.719 0.000 0.029 
Model for BASEMATH slope (B8) 
Intercept (G80) 0.180 0.037 4.836 0.000 0.011 
Random Effects Variance df Chi-Square 
(Var. Components) (p) 
Var. in school means (U0) 7.533 53 awe 
(0.004) 
Var. within schools (R) 157.261 
From Unconditional Model 
Random Effects 6 Chi-Square 
(Var. Components) Variance df p) ICC 
Var. in school means (U0) 8.713 62 oi) 0.032 
Var. within schools (R) 263.599 
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Table R-21 


(Analytic Group 10, Final Model) 


Model 26: three-year program impact on 6th-grade Tier 3 students main effect model 


Fixed Effects Coefficient SE t p Glass’s A 

Model for INTRCPT1 (BO) 

Intercept (G00) 226.691 0.712 318.327 0.000 -- 

PREAD (G01) 12.842 4.706 2.729 0.009 0.791 

PSPED (G02) 17.357 10.213 1.699 0.094 1.069 

TRT (G03) 0.044 1.005 0.044 0.966 0.003 
Model for GENDER slope (B1) 

Intercept (G10) 2.426 0.724 3.351 0.001 0.149 
Model for IEP slope (B2) 

Intercept (G20) -5.453 1.515 -3.600 0.001 -0.336 
Model for LEP slope (B3) 

Intercept (G30) -11.099 1.708 -6.500 0.000 -0.683 
Model for BASEISAT slope (B4) 

Intercept (G40) 0.460 0.044 10.485 0.000 0.028 
Model for BASEMATH slope (B5) 

Intercept (G50) 0.176 0.035 4.976 0.000 0.011 

Random Effects Narinnce df Chi-Square 

(Var. Components) (p) 

Var. in school means (U0) 5.920 59 S510 
(0.014) 
Var. within schools (R) 157.476 
From Unconditional Model 
Random Effects ; Chi-Square 

(Var. Components) Variance df p) ICC 
Var. in school means (U0) 8.713 62 om 0.032 
Var. within schools (R) 263.599 
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Table R-22 


(Analytic Group 10, Full Model) 


Model 27: three-year program impact on 6th-grade Tier 3 students interaction model 


Fixed Effects Coefficient SE t p Glass’s A 
Model for INTRCPT1 (BO) 
Intercept (G00) 226.850 0.701 323.484 0.000 -- 
PMIN (G01) 15.286 13.161 1.161 0.251 0.941 
SIZE (G02) 0.001 0.004 0.141 0.889 0.000 
PREAD (G03) 13.306 5.977 2.226 0.030 0.819 
PFEM (G04) -3.641 10.421 -0.349 0.728 -0.224 
PSPED (G05) 18.961 10.996 1.724 0.090 1.168 
PLEP (G06) 10.496 12.774 0.822 0.415 0.646 
PLUNCH (G07) 0.406 16.977 0.024 0.981 0.025 
COHORT (G08) -1.938 1.587 -1.221 0.228 -0.119 
TRT (G09) -0.009 1.072 -0.008 0.993 -0.001 
Model for GENDER slope (B1) 
Intercept (G10) 0.355 0.870 0.408 0.683 0.022 
TRT (G11) 4.355 1.329 3.278 0.001 0.268 
Model for IEP slope (B2) 
Intercept (G20) -6.054 1.506 -4.020 0.000 -0.373 
TRT (G21) 1.245 2.828 0.440 0.659 0.077 
Model for LEP slope (B3) 
Intercept (G30) -10.826 1.868 -5.796 0.000 -0.667 
TRT (G31) -0.647 2.366 -0.273 0.785 -0.040 
Model for LUNCH slope (B4) 
Intercept (G40) 0.101 3.373 0.030 0.976 0.006 
TRT (G41) -5.576 5.531 -1.008 0.314 -0.343 
Model for BLACK slope (B5) 
Intercept (G50) -0.514 5.387 -0.095 0.924 -0.032 
TRT (G51) 2.263 5.536 0.409 0.682 0.139 
Model for HISPANIC slope (B6) 
Intercept (G60) -2.667 4.968 -0.537 0.591 -0.164 
TRT (G61) 1.529 5.435 0.281 0.778 0.094 
Model for BASEISAT slope (B7) 
Intercept (G70) 0.503 0.055 9.206 0.000 0.031 
TRT (G71) -0.078 0.084 -0.933 0.352 -0.005 
Model for BASEMATH slope (B8) 
Intercept (G80) 0.121 0.059 2.032 0.042 0.007 
TRT (G81) 0.113 0.071 1.589 0.112 0.007 
Random Effects Warinice df Chi-Square 
(Var. Components) (p) 
Var. in school means (U0) 7.511 53 aie 
(0.006) 
Var. within schools (R) 157.112 
From Unconditional Model 
Random Effects : Chi-Square 
(Var. Components) Variance df p) ICC 
Var. in school means (U0) 8.713 62 ne 0.032 
Var. within schools (R) 263.599 
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Table R-23 


Model 28: three-year program impact on 6th-grade Tier 3 students interaction model 
(Analytic Group 10, Final Model) 
Fixed Effects Coefficient SE t Pp Glass’s A 
Model for INTRCPT1 (BO) 
Intercept (G00) 226.637 0.716 316.468 0.000 -- 
PREAD (G01) 13.078 4.668 2.802 0.007 0.805 
PSPED (G02) 16.559 10.185 1.626 0.109 1.020 
TRT (G03) 0.069 1.007 0.069 0.946 0.004 
Model for GENDER slope (B1) 
TRT (G11) 4.193 1.026 4.085 0.000 0.258 
Model for IEP slope (B2) 
Intercept (G20) -5.394 1.492 -3.615 0.001 -0.332 
Model for LEP slope (B3) 
Intercept (G30) -11.474 1.613 -7.114 0.000 -0.706 
Model for BASEISAT slope (B4) 
Intercept (G40) 0.463 0.043 10.701 0.000 0.028 
Model for BASEMATH slope (B5) 
Intercept (G50) 0.174 0.036 4.866 0.000 0.011 
Random Effects Warianie df Chi-Square 
(Var. Components) (p) 
Var. in school means (U0) 5.909 59 Bose 
(0.013) 
Var. within schools (R) 156.767 
From Unconditional Model 
Random Effects ; Chi-Square 
(Var. Components) Variance df p) ICC 
Var. in school means (U0) 8.713 62 oo) 0.032 
Var. within schools (R) 263.599 
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